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EXECUTIVE SUMMARY 

Prevention of drug, alcohol, and tobacco abuse among high risk youth has been the goal of the 
Center for Substance Abuse Prevention (CSAP) since 1987. In October 1994, CSAP granted 
funding to the Chrysalis Project in Portland Public Schools to prevent or delay the onset of 
substance abuse among a special target population — high risk female adolescents with a 
history of childhood abuse. For many of these young women, a strong sense of shame related to 
this abuse impacts their self-esteem and can lead to problems, such as substance abuse, eating 
disorders, poor social relationships, violence, and suicidal tendencies. 

The Chrysalis Project is designed to serve high school aged young women who have been 
victims of physical, sexual, or emotional abuse. The project provides an integrated matrix of 
services to reduce the negative consequences of this, abuse and to strengthen the girls’ own 
resilience. Among the key intervention strategies of the project are weekly student support 
groups, educational Open Sessions, a Challenge Course, a Girls Empowerment physical safety 
and self-defense class, an All Student End-of-Year Celebration, and ongoing individualized Case 
Management. Through these activities, students encounter emotional support and high 
expectations from caring adults. 

This report presents the findings from the evaluation of the Chrysalis Project’s second year of 
providing assistance to students. During the 1995-96 school year, the project served 370 young 
women in grades nine through twelve (189 Chrysalis Program girls and 181 control group youth). 
The program operates in all ten traditional high schools and two alternative schools in the district. 

The Chrysalis evaluation utilizes a true experimental design with random assignment of youth to 
intervention and control groups and pre and posttesting of subjects. The purpose of this 
evaluation is to assess the effectiveness of the specific program intervention strategies. It will 
also document the process of service delivery and program implementation at the schools to help 
interpret and give context to the project outcomes. 

Project Chrysalis enjoyed much success at its 12 school locations during its second year of 
operation. The key findings of the outcome evaluation indicate several significant relationships 
among different health risk behavior areas, including: 
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■ Chrysalis students attending more support groups reported lower rates of 
marijuana use in the past 30 days; 

■ Chrysalis students attending more support groups reported fewer instances of 
sexual intercourse in the past 3 months; 

■ Students who participated in more Chrysalis activities (high dosage) reported 
more negative attitudes toward alcohol, tobacco, and other drug use and 
stronger refusal skills; 

■ Students who participated in more Chrysalis activities (high dosage) 
significantly improved their academic indicators, such as higher Graduation 
Standards Test (GST) scores, higher grade point averages, and reduced 
school absenteeism; 

■ Over the course of the year, fewer Chrysalis Program group students became 
pregnant than Research students; 

■ A higher percentage of Chrysalis girls reported using alcohol and marijuana in 
the past 30 days, while the percentages remained essentially the same in the 
Research group; 

■ Over the school year, fewer students in both groups engaged in physical 
fighting. While the main effects of fighting go down throughout the high school 
years. Chrysalis students had bigger declines than would be expected in 
fighting; 

■ Chrysalis students receiving more case management sessions and hours 
were more likely to have carried a weapon to school in the past 30 days; and 
more likely to have lower GPA and school attendance; and 

■ Students at the highest risk level (8 or more risk factors at intake) reported: 

» higher use of cigarettes and marijuana at school, 

» higher rates of sexual intercourse with more partners, 

» higher rates of considering, planning, and attempting suicide. 



These results indicate just how dramatically experience can stimulate positive or negative 
behavior in young women. These, and related findings, have taught the researchers much about 
the implementation, management, and evaluation of the project this year. However, it is 
important to note that all of the results at this stage of the project are merely associations; that is, 
they show the presence or lack of related behaviors and not the cause of the behavior. 



The data in this report offer a striking portrait of the Chrysalis high risk female adolescent 
population. Clearly, the results show that relationships exist between young women with a 
history of abuse and increased use of alcohol and other drugs, sexual behaviors; violence- 
related behaviors, and suicide ideation and attempt. Despite lower enrollment than anticipated. 
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staff changes, and other limitations, the project has shown remarkable resiliency and maintained 
steady progress toward identifying problems and framing solutions. While recognizing this 
success, the authors recommend continued attention be given to schools in which the project 
has not been fully implemented, so that any obstacles to implementation can be overcome. 

In summary, while no single approach will work for all students, the Chrysalis demonstration 
project has offered a variety of options and support services to address multiple health risk 
behaviors associated with the high risk female adolescent population. Chrysalis is a place where 
students can develop competence and confidence in their abilities, where they can have 
successful experiences and learn constructive behaviors, where they can find a passage to a 
safe future. We anticipate that with each passing year there will be increasing evidence of the 
positive impact the program has on the lives of students. 



ERIC 



Page ix 



Portland Public Schools • RMC Research 



Chapter 1. Overview of the Intervention 

Adolescents today face more complex and life-threatening issues than young people faced only a 
generation ago. Health risks have changed dramatically in the last decade due to AIDS, drug 
addiction, child abuse, teen pregnancy, gangs, and violence. Increasingly, female adolescents are 
involved in unhealthy risk taking and self-destructive behaviors — suicide and suicide attempts, 
sexual intercourse, tobacco use, and drug abuse. Yet, the picture for young adolescents is not 
entirely bleak. Research tells us that a single caring adult can make all the difference in the direction 
a young adolescent woman chooses to take in overcoming these formidable obstacles in her life. 

Statement of the Problem 

At the national level, the need to address one of these issues — student alcohol and other drug use 
and its consequences — has led to a variety of program and policy initiatives. In order to meet the 
needs of the high risk female adolescent population, the U. S. Department of Health and Human 
Services’ Center for Substance Abuse Prevention initiated the High Risk Youth Demonstration Grant 
Program. This program helps local and state agencies identify current health and safety habits so 
that improvements can be made where they are needed and effective practices can be demonstrated 
and replicated throughout the United States. The Portland Public Schools’ Prevention Office 
received a grant to demonstrate the effectiveness of a comprehensive model designed to reduce the 
negative consequences of childhood abuse among high school aged young women. 

I' 

The young women participating in the Chrysalis Project have all been victims of abuse. Statistics 
on the rates of child abuse are controversial. Conservative estimates of incidents involving physical 
contact find that approximately one in three girls is sexually abused before age 18 and one in four 
by age 14 in the United States (Hopper, 1996; Russell, 1993). The rates of physical abuse and 
emotional maltreatment among these young women are even higher. Most abused children never 
come to the attention of authorities because there may be no physical signs of harm, victims often 
have intense shame, and even adults who know of the abuse deny the truth for fear of destroying 
the family. The effects of child abuse have also been linked to delaying cognitive development and 
inhibiting healthy physiological responsiveness to the environment (Carrey, 1995). Most of the 
Chrysalis participants show ongoing signs of posttraumatic stress disorder (intrusive recollections, 
constricted affect, etc.) and other chronic trauma-related disorders. 
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Brief Description of the Intervention 

The Chrysalis Project in Portland Public Schools serves high school aged young women who have 
a history of physical, sexual, or emotional abuse. Chrysalis is a comprehensive, multifaceted project 
designed to reduce the negative impact of this eibuse and enhance the healthy, positive development 
of the young women. All the female adolescent participants in Chrysalis, in both Program 
(intervention) and Research (control) groups, are victims of abuse. These students exhibit a wide 
range of other risk factors including being a child of a substance abuser, experiencing chronic school 
failure, dropping out of school, being pregnant or a teen parent, using alcohol and other drugs, 
experiencing mental health problems, showing violent/delinquent behaviors, being at risk of suicide, 
HIV/STD infection, or other threats to their health. 

Based on research of effective interventions for high risk youth. Project Chrysalis offers multiple 
intervention strategies to provide intensive services and comprehensive support systems (Klitzner, 
1987; Hawkins et al., 1986; DeCiccio et al., 1984; Weisheit, 1983) that address risk and resiliency 
factors across several domains. Resilience research offers hope that internal characteristics and 
interpersonal relationships may buffer the negative influences that occur in young lives and influence 
their positive life adaptations (Laws & Gabriel, 1995; Benard, 1986). 

Chrysalis provides an integrated web or matrix of services to foster resiliency characteristics of 
students. Through interventions such as the weekly Chrysalis support group, students encounter 
emotional support and high expectations from caring adults. Open sessions are scheduled 
periodically to augment students’ knowledge of specific issues such as HIV/AIDS, media literacy, 
or tobacco awareness. Intensive interventions such as the Challenge Course, Girls Empowerment, 
and End-of-Year Celebration are once a year milestones which positively impact the adolescents’ 
sense of self-concept, personal power, and autonomy. These interventions influence the individual, 
peer, school, and community domains. 

Theoretical Underpinnings Linking the Intervention to Desired Outcomes 

The intervention is designed to reduce the incidence and prevalence of alcohol, tobacco, and other 
drug use and related problems among high-school-aged young women who have experienced 
physical, sexual, or emotional abuse. This population was selected because of the strong correlation 
between a history of physical/sexual abuse and involvement with various chemical substances. 
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For many of these young women, a strong sense of shame and degradation related to sexual abuse 
in their family can lead to problem behaviors such as eating disorders, low self-esteem, substance 
abuse, aggression, and poor relationships with peers (Downs, 1993). Victims of physical abuse 
suffer similar effects, especially the loss of basic trust in relationships (Hoffman, 1995). Female 
survivors of abuse may have increased risks of drug involvement with the conflicting pressures and 
developmental changes of adolescence. As the term Chrysalis suggests, the project provides a 
special protective cocoon for young women at a vulnerable developmental time to urge them toward 
a healthy transition to adulthood. 

Hypotheses Relating Intervention Activities to Measured Change 

Exhibit 1.1 on the following page shows the Chrysalis Logic Model. The model illustrates in more 
detail the relationships among the risk factors identified in the target population, the interventions 
implemented by the project, and the outcomes hypothesized for the female adolescents as a result 
of their participation in the project. Each intervention addresses more than one risk factor and is 
expected to lead to more than one positive outcome for the young women. The assumption 
underlying the project is that by providing a caring and supporting environment though the various 
program interventions, the project will demonstrate improved school performance, reduced negative 
impact of child abuse, and strengthened resiliency. These factors will help the Chrysalis participants 
to reject drug use, bolster their personal strengths, and build strong, supportive networks for a 
positive future. 
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Exhibit 1.1 Chrysalis Logic Model 



F 


^ISK FACTORS 


OBJECTIVES 


INTERVENTIONS 


INTERMEDIATE 


IMPACT 












OUTCOMES* 






Poor academics 


To significantly improve 




SKILL BUILDING 


improved school 


• Reduced incidence 


5 




school performance 




KNOWLEDGE 


performance as 


and prevalence of 


I 




among high school 




ADVOCACY 


measured by: 


ATOD use and 




Lack of school 


female participants in 




PEER SUPPORT 


# improved 


related negative 




bonding 


Project Chrysalis 






j attendance 


consequences 




♦ 


Support Group 


• reduced 












dropout rates 
# progress in 


• Increased 




Poor life skills 


' 


♦ 


Challenge Course 


perception of harm 












meeting 


& peer disapproval 








♦ 


Open Sessions 


graduation 


associated with 




Poor self image 






Drug Awareness 
Tobacco Awareness 
HIV/AIDS Education 
Women Role Models 
Goal Setting 


standards 

Reduced negative 
consequences of 


ATOD use 

• Reduced 
involvement in 
physical fighting, 




Abuse/sexual 

abuse 












Eating Disorders 


child sexual abuse 


victimization, 






To significantly 




Coping Skills 


as measured by: 


assault injuries, 




Cultural gap 


decrease the incidence 




Friendship Skills 


• information/ 


aggressive 


O 

P 


between school 


of negative outcomes of 




Media Literacy 


statistics on 


behavior, weapons 


§ 


and students 


child physical/ sexual 




Balancing: 


unsafe sexual 






and/or community 


abuse among high 




work/family/school 


practices 


• Reduced school 




Lack of activities 


school female 






# reduced early 


dropout, early 




and attitudes that 


participants in Project 


♦ 


Case Management 


pregnancy 


pregnancy and 




promote school 


Chrysalis 




Needs Assessment 


# reduced STD's 


parenthood. HIV/ 




bonding 






Individual Service 


# reduced mental 


AIDS infection & 


to 


Rejection by 






Plan 


disorders 


sexually transmitted 




peers/isolation 






Opportunities Fund 




diseases, mental 


o. 




To significantly 
strengthen resiliency 
among high school 
female program 




Portfolios 


Improved resiliency 
as measured by 
resiliency scales 


disorders, such as 


fc 

1 


Lack of successful 
role models 


♦ 


Girls Empowerment 


eating disorders, 
depression, and 
suicidal behaviors 


1 




participants 


♦ 


School Staff Training 


on the Chrysalis 




Lack of economic/ 






in Sexual Abuse 


Risk St Resiliency 






employment 

opportunities 








Index 





• See Evaluation Plan for complete list of outcomes. 




Page 4 



IS 



Portland Public Schools ■ RMC Research 



Chapter 2. Implementation Summary 



Program Description 

The Chrysalis Project serves young women in grades nine to twelve with a history of physical, 
sexual, and/or emotional abuse with a comprehensive, school-based program to reduce ATOD 
abuse, improve school performance and self-image, and reduce negative consequences of child 
abuse. During Year Two, a total of 370 high-risk female adolescent youth participated in the project; 
189 Program group (intervention) students and 181 Research group (control) students. 

Recognizing the high-risk status of female survivors of childhood abuse. Project Chrysalis has woven 
together a multifaceted strategy to address this population’s multiple and interrelated problems. As 
the project’s name suggests. Chrysalis is designed to address female students at a vulnerable 
developmental stage — the adolescent high school years — and to coax them toward a dramatic 
change or metamorphosis. The project is designed to counter risk factors with protective factors, 
to replace fear and isolation with confidence and supportive networks, to bolster knowledge, and 
advocate for student needs. Together the project interventions described below comprise a 
comprehensive, state-of-the-art system to guide high risk female adolescents on a road to recovery 
from child abuse. 

I 

Student Support Groups 

School-based student support groups are the core of the Chrysalis Project with other interventions 
radiating from and enhancing this central component. In the second year of the program, each of 
the twelve participating high schools served up to 20 young women who had experienced physical, 
sexual, and/or emotional abuse, wanted to participate in the support group, and had the necessary 
parent/guardian permission. Five schools established and maintained one support group of 
approximately 10 students; seven schools established two concurrent weekly support groups. 

Each support group is co-facilitated by two trained adults; a school counselor/case manager and a 
child abuse therapist from Waverly Children’s Home. Co-facilitation relieves the stress of working 
with this challenging student population, expedites information exchange, exposes the young women 
to positive teamwork and interactions, and provides a balance between individual strengths and 
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styles. Support groups are held during the school day to encourage regular attendance by the girls. 
The curriculum for the support groups was developed for the adolescent female population by 
Waverly Children’s Home. An outline of its contents is included in this report as Appendix A. The 
early sessions define what a “chrysalis” is, go over the group’s purpose, establish ground rules, and 
introduce some techniques for establishing personal privacy, safety, and confidentiality. Over the 
course of 24 sessions, students discuss and examine a wide range of topics including: 

■ Child Abuse: Definitions, Causes and Effects; 

■ Telling, Understanding, and Reclaiming My Story, including secrets, not talking, memory, and 
family dynamics; 

■ States and Conditions Specific to Abuse Trauma, including contaminated thinking, depression/ 
suicide, shame/guilt and emotional flooding; 

■ World Views and How They are Impacted by Abuse; 

■ Past Coping Strategies For Survival, including alcohol/drug use, sexual activity, hurting self or 
others, perfectionism, and introducing new coping strategies; 

■ intimacy and Sexuality, including the impact of abuse; 

■ Attitudes About Women, including celebrating our power; and 

■ Closure Process, including techniques and referrals for continuing the recovery process. 
Open Sessions 

Periodically throughout the school year, a special educational “open session” is held to address the 
specific needs and interests of support group participants. These sessions provide a break from the 
intensive personal exploration required in the support groups. They are intended to “open” the 
participants to new skills, knowledge, networks, and opportunities. These educational sessions 
focus on topics related to important risk and resiliency factors. 

During Year Two, the project received two supplementary grants from CSAP to provide HIV/AIDS 
education and media literacy training to the Project Chrysalis participants. These two supplemental 
programs replaced the planned Open Session topics in Year Two. Examples of the Year Two Open 
Session activities are: four HIV/AIDS education classes and three media literacy training classes. 
In subsequent years, the project will return to the proposed Open Session topics: alcohol and other 
drug prevention; tobacco awareness; eating disorders; nutrition; goal setting; career interests; 
positive women role models; and coping skills to achieve a healthy, balanced life. 
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Case Management Services 

Individualized case management ensures that each student’s needs are linked with the appropriate 
district or community resources. In each high school, a counselor or other appropriate trained staff 
serves as the Chrysalis case manager to co-facilitate the student support groups and serve as an 
ongoing advocate for students. Each student completes a brief needs assessment to identify the 
major issues and concerns in her life. Case managers use the needs assessment to develop an 
individual service plan which addresses these issues of concern. Each case manager has a small 
opportunity fund allotted for emergency situations with students and access to special programs to 
support student needs, including food, clothing, academic tutoring, bus fare, fees to attend a tobacco 
cessation class, etc. 

Challenge Course 

The Challenge Course invites Chrysalis intervention participants to test their cooperative group 
problem solving, goal setting, and break-through thinking in an exhilarating physical obstacle course. 
The course, conducted by Maryann “Moe” Carrick, Director of the Northwest Regional Office of 
Project Adventure, involves students in safe but demanding physical and group activities. During 
the course, students build a sense of trust, teamwork, and personal accomplishment. The Challenge 
Course is a blend of theory and practice. Meaningful adventure-based learning experiences 
designed to develop safety and trust within a group are followed by structured dialog sessions where 
students process their experiences and make connections between behavior and recovery. Many 
Chrysalis program participants have trouble concentrating in school and are uncomfortable with 
physical activities and with asserting themselves. The Challenge Course offers an effective antidote 
to these problems with activities which are structured to complement the goals of the program. 

Girls Empowerment Physical Safety and Self-Defense Class 

The Girls Empowerment course is designed to improve the young women’s confidence, self-image, 
and sense of personal competency and autonomy by teaching them how to protect and assert 
themselves. The class is led by women who have attended the Portland Police Bureau’s 80-hour 
Women Strength course. Using a variety of instructional methods ranging from instructional 
presentations and group discussions to role playing and physical exercises, participants learn about 
the following topics: assertiveness and street safety, date rape, domestic violence and distinguishing 
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consensual from offensive acts, gender stereotypes and myths about violence to\ward \women, 
protecting oneself by a\wareness of the surrounding environment, assertive speech and actions, 
identifying escape options, and trusting feelings about uncomfortable or unsafe situations. During 
Year T\wo of the project, the girls empo\werment curriculum \was held as a one-day course during the 
school day. With the aim of improving the level of participation, the class \was held at a Portland 
Public Schools gym using special protective equipment and supplies for the physical safety class. 

All-Student Year-End Celebration 

At the end of the school year, all of the teen \women \who have participated in the Chrysalis Project 
get together for a half-day celebration and final healing experience. Throughout the school year 
students have met primarily in their small support groups. The Year-End Celebration is an 
opportunity for students to experience being part of the larger group and celebrates their rite of 
passage on the road to recovery from childhood physical, sexual, and emotional abuse. Activities 
are planned by the students arid include \women’s theater performances, poetry readings, 
inspirational guest speakers, exhibits of artwork, music, dance, and positive recreational 
opportunities. Small group activities encourage the young women to get to know each other and 
share their impressions of the preceding year’s activities. Certificates of achievement and awards 
recognize the young women’s efforts and participation. The Year Two retreat was held at a local 
school-community center during the school day so that all young women could participate. 

Supplementary Grant and Carryover Activities 

The Chrysalis Project received two CSAP supplementary grants during the second year of the 
program — an HIV/AIDS Education Project and a Media Literacy Project. The activities of these two 
special programs were infused into the existing Chrysalis Open Sessions curriculum. Another 
special project, the Validity Study of Student Drug Use, was conducted with carryover funds. The 
complete reports on these special activities are presented in Appendices B, C, and D. The following 
section provides an overview of the implementation of these supplementary program activities. 

The Validity Study of Self-Reported Drug Use 

The Chrysalis project received CSAP funding to conduct a special study to validate the self-report 
of certain health risk behaviors on the project's Risk and Resiliency Index survey. The study 
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features the use of hair specimen analysis as a validating measure of students' self-reports of their 
illicit drug use. As always, results of the hair analysis and the self-report survey were kept 
completely confidential. 

Surveys of alcohol and other drug use are often criticized by policymakers and the general public 
as relying too heavily on the honesty of the respondent. The National Institute on Drug Abuse 
(NIDA) has identified other sources of error that can be equally biasing (Turner et al, 1992) in the 
collection of this information via self-report. Young people's reading levels, comprehension of 
complex concepts within drug use items, and attention spans — in what are often very long surveys 
— are other complicating factors having nothing to do with their willingness to disclose or respond 
honestly to these questions. 

Project Chrysalis aspires to reduce ATOD use behaviors among its participating young women, and 
information regarding these behaviors was collected via a self-report survey. To lend confidence 
to the interpretation of these results, a validation study was designed and implemented by project 
evaluation staff. An overview of the results of the study will be presented in Chapter 4 of this report. 
Readers interested in the details of the research design, data analysis, and results of the study are 
directed to read the complete report in Appendix B, The Validity Study of Self-Reported Drug Use. 

HIV/AIDS Education Project 

The Chrysalis Project implemented an HIV/AIDS Education course with Program and Research 
group youth during Year Two. An HIV/AIDS Prevention Education Specialist worked with public and 
private community organizations, such as the Cascade AIDS Project, to develop a comprehensive 
and informative educational experience for Chrysalis participants. The HIV/AIDS curriculum was 
delivered in four separate sessions to Program girls and in a single retreat setting to Research youth. 
Both groups receiyed the same comprehensive educational program. The sessions provided 
concrete, interactive activities and lessons on the following topics: classroom climate setting; what 
is HIV and what are HIV high-risk behaviors; testing for HIV; sexual abstinence and protection from 
pregnancy, HIV, and other sexually transmitted diseases; refusal and negotiating skills; avoiding 
high-risk situations; and information for accessing/using resources for HIV protection. 

Two student trainers conducted most HIV/AIDS education activities in a “Teen to Teen” Workshop. 
A unique aspect of the session was having the delivery and responsibility for the program given to 
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teenagers. This type of presentation is thought to be effective because teens are more comfortable 
discussing sensitive issues surrounding sexuality with their peers. Another powerful component of 
the program involved having speakers who were HIV-positive talk to the youth. These speakers, 
who were also victims of childhood sexual abuse, spoke about their experience of getting the HIV 
virus and how the virus has changed their lives. Their stories gave a personal focus to a virus that 
doesn't seem a real threat to many young girls because they don't think it can happen to them. As 
one speaker poignantly stated, HIV has really taken over my life. A full description of the HIV/AIDS 
Education Program is in Appendix C. 

Media Literacy Project 

The Chrysalis MEDIA Project was designed to empower students to think critically about the 
information they receive in a multimedia society and the decisions they make daily based on 
advertising and other media influences. 

The MEDIA Project consisted of three major components: three training sessions for all Chrysalis 
Program students during the school year; a production workshop for a select group of ten students 
during the summer; and a proposed training retreat for ten Chrysalis students to develop an 
educational program for middle school students. 

Three, one-hour Open Sessions led by a woman media specialist were offered for all Chrysalis 
Program participants. The sessions, held at the twelve high schools, examined: 1) gender roles and 
stereotypes found on television and in other media, including attitudes toward various cultural groups 
and specific female populations — for example, disabled, elderly, or poor women; 2) how to "read" 
an advertisement or other media for the underlying message being disseminated; and 3) discerning 
fact versus fiction in the media — encouraging students to become analytical and critical thinkers. 

In June 1996, ten Chrysalis students were recruited to apply what they had learned about media 
literacy during the school year to designing and producing a public service announcement (PSA) and 
a short videotape. The students attended a two-week media literacy workshop led by two 
professional female animators/film makers from the Northwest Film Center's Video/Film Maker-in- 
Schools Program and by Chrysalis staff. The use of animation allowed participants to avoid 
confidentiality issues while addressing important subject matter in a creative manner. Students 
created two products: a 30-second PSA entitled Fight Girl Poisoning and a three minute videotape. 
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Girl Power. The participants determined the content of the two products; staff choose an animated 
film format to address the issues of media influence, ATOD use, gender and abuse and to protect 
participants’ privacy. Each girl played a role in the development and production. Some developed 
story lines about an ATOD issue important to them, others drew the pictures or wrote the script and 
audio. The girls learned what animation is, what it entails, and how to develop their own animation. 

During the 1996-97 school year, the workshop participants will deliver presentations on media 
literacy and show the animated Fight Girl Poisoning PSA in middle school health classes. The 
students will also help arrange other opportunities to air the PSA and encourage new Chrysalis 
participants to become involved in educating their middle school peers. A detailed description of the 
MEDIA Project, along with evaluation results, is presented in Appendix D. 

Training for School Staff 

Another aspect of the Chrysalis intervention focuses on the cultural divide and the resulting 
alienation often experienced by abused students toward school. Training provided by the Waverly 
Children’s Home and the Tribes Learning Community at the district high schools improved the ability 
of school staff to address issues related to physical/sexual abuse, cultural diversity, and cooperative 
learning among students. During 1995-96, a Waverly contractor provided five 90-minute orientation 
sessions on the trauma of child physical, sexual, and emotional abuse among female adolescents 
to the faculty at four high schools. Fifty-seven school staff members attended these sessions. The 
Waverly contractor also presented an advanced training. Understanding Secondary Trauma, to 20 
project staff and case managers in January 1996. A district Tribes trainer presented one 32-hour 
course designed to enhance multicultural understanding among staff from each high school. The 
Tribes course shows participants how to use developmentally and multiculturally appropriate 
cooperative learning and peer support strategies to improve the climate for learning, student self- 
esteem, and success in school. 

In Year Two, Project Chrysalis also sponsored a 2V2 day Challenge Course retreat for staff and case 
managers. The retreat was held at the Tilljkum Retreat Center, a Project Adventure training facility 
sponsored in conjunction with George Fox College. Thirteen Chrysalis school-based staff attended 
the session focusing on experiential learning and strategies to build trust, teamwork, and personal 
accomplishment. Special attention was given to linking knowledge acquisition to knowledge 
utilization. The case manager’s personal experience in challenge course activities allowed them to 
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feel the success and/or frustration students encounter in these physical activities. Following each 
challenge course activity, the school staff brain stormed debriefing methods and exercises to extend 
the impact of the curriculum with Chrysalis students. 

Linkage to National Cross-Site Evaluation 

During Year Two, Project Chrysalis was selected to participate in CSAP’s National Cross-Site 
Evaluation Study. The Chrysalis project management team collaborated closely with Senior 
Evaluators at EMT Associates and Macro International to implement a high quality system for 
tracking “dosage” or student participation in program activities. Based on the CSAP management 
information form (MIF) categories, Project Chrysalis developed an electronic database system to 
provide the project with systematic access to student information from multiple sources, enhance 
student tracking and data management, and mesh with data collection efforts of the national cross- 
site evaluation. 

For Year Three, the project case management logs have been revised to incorporate the latest 
intervention categories and coding systems provided to the Chrysalis staff and the EMT 
Associates/Macro International local data collector at the National Cross-Site Evaluation Training 
Conference in St. Louis, Missouri held in July 1996. 

Dosage of the Intervention 

Year Two of the Chrysalis Project, the 1995-96 school year, was the first full year implementation 
of the program and allowed the delivery of a full complement of program services. This was the last 
year of intensive program services for cohort one students and the project benefited by having this 
continuing group of students with which to begin the second year of the program. The schools 
conducted an ongoing student intake between September and December 1995. Support groups 
began in late September 1995. Throughout Year Two, the management information system in the 
PPS Evaluation Department tracked student participation or dosages in the project. 

Exhibit 2.1 below compares an “ideal” full-year implementation configuration for Project Chrysalis 
with the “actual” level of program implementation in Year Two. Project Chrysalis provides students 
with individual case management, as well as five key program components — Support Groups, Open 
Sessions, a Challenge Course, a Girls Empowerment class, and a Year-End Celebration. The ideal 
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program involves 24 Support Group sessions, 7 Open Sessions, (four HIV/AIDS education and three 
media sessions), one Challenge Course, one Girls Empowerment course, and one all-student Year- 
End Celebration. This ideal program would provide 34 weekly sessions or approximately 40 hours 
of intervention activities, and up to 10 hours of individual case management annually. 

During the second year of the program. Project Chrysalis provided a mean of 29.6 sessions or 87 
percent of the optimal program services (based on 34 program sessions for full implementation). 
The mean implementation level of Support Groups across the program was 84 percent or an average 
of 20.2 sessions per school during the 1995-96 school year. The implementation rate for Open 
Sessions was 93 percent or a mean of 6.5 sessions. The rate of implementation for the Challenge 
Course was 90 percent with one alternative school not delivering this component. Both the Girls 
Empowerment and the Year-End Celebration had 1 00 percent implementations by the project. 



Exhibit 2.1 

Chrysalis Project, Year Two 

Comparison of Ideal vs. Actual Program Implementation 
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Exhibit 2.2 compares the percentage of proposed ideal program dosages with the actual level of 
program implementation by component during Year Two. The ideal implementation configuration 
proposes that 48 percent of the intervention consists of Support Group sessions, 15 percent of the 
program is the Girls Empowerment course, 14 percent of the project are Open Sessions, 12 percent 
of the program is the Challenge Course, and 10 percent is the student Year-End Celebration. 



In its second year, the project achieved full implementation across all schools. Forty-one percent 
of the intervention hours were comprised of Support Group sessions, 13 percent of the hours were 
each devoted to the Girls Empowerment and Open Sessions, 12 percent of the program hours were 
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for the Challenge Course, and 10 percent of the hours were for the Year-End Celebration. Only 10 
percent of the ideal project activity hours remain to be implemented in the future (seven percent 
Support Groups, two percent Girls Empowerment, and one percent Open Sessions). 



Exhibit 2.2 

Chrysalis Program Dosage, Year Two 
Level of Implementation by Component 





Support Groups 
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1 Year End Celebration 
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Exhibit 2.3 on the following page provides a summary of the implementation of Project Chrysalis at 
each of the 12 high schools during Year Two. A total of 19 Project Chrysalis groups were led by 13 
school-based case managers, in collaboration with six contract therapists from Waverly Childrens’ 
Home. The matrix also provides information on attrition of students in the Program (intervention) and 
Research (control) groups. Approximately 39.6 percent of the program students (n=75) and 23.7 
percent of the research students (n=43) were described as “inactive” at the end of Year Two. 
Students are “inactive" if they move out of the service area, do not recommit to attend program 
activities, drop out of school, or are lost after multiple attempts to contact them by mail or phone. 
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Exhibit 2.3 

Chrysalis Implemen tat ion Across Schools-and.Proqram. Year Two (1995-9 6). 
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During Year Two, all case managers at the twelve high schools achieved full implementation of the 
Chrysalis Project (as defined by implementation of at least one-third of each of the program 
components). The mean level of program implementation across schools was 29.6 sessions 
delivered during 36 hours; the optimal implementation of the program is 34 sessions and 40 hours 
annually. A comparison of the ideal implementation hours vs. the actual implementation indicates 
that 90 percent of the intervention hours were achieved by the schools in Year Two. This is a 
significant improvement in the level of implementation over Year One and an appropriate level of 
implementation given the expansion of the program to four new schools in the second year. Analysis 
of program dosages by school indicates the number of Chrysalis sessions provided in the schools 
ranged from 23 to 35 sessions or approximately 68 to 100 percent of the “ideal” implementation. 

In addition to the five key components. Chrysalis case managers conducted individualized case 
management activities and one-to-one counseling with students to improve self-image and academic 
performance and to reduce the negative consequences of childhood abuse. Case management is 
a dynamic activity and encompasses several types of activities in the schools. During Year Two, 
the case management categories were; student intake, consent/recommit to participate, one-to-one 
counseling, academic support, follow-up to support group topics, opportunity fund (incentives for 
participation), crisis intervention, communication with school staff, vocational support, and other 
services. The other category included individual needs assessment, participation by students in 
program awareness/dissemination activities, and behavioral/social skills counseling. The case 
management instrument was revised for Year Three to link more closely with the CSAP management 
information form (MIF) categories and the National Cross-Site Evaluation intervention categories. 

During Year Two, case managers at each school logged a mean of 165 case management contacts 
with Chrysalis Program students and 55 contacts vt^h Research (control group) students — sporadic 
contacts with control group students helped to insure against attrition among this group. Case 
management also included contacts with parents (10 percent), school staff (7 percent), and 
community agencies (3 percent). The number of case management hours per Chrysalis Program 
student ranged from 0.2 to 9.2 hours annually. Overall, Chrysalis Program group participants each 
received an average of 3.9 case management hours during the second year of the project. 

Exhibit 2.4 displays the type and hours for case management activities in 1995-96. The twelve 
Chrysalis schools provided a total of 1202 case management hours with participants. Case 
managers reported 145 hours for student intake and 32 hours for collecting consent forms from 
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participants and parents. Counselors spent 123 hours with students on case management related 
to academic support; activities to follow up on support group topics took 119 hours; one-to-one 
counseling sessions involved 214 hours; and 155 hours were used for crisis intervention activities. 



Exhibit 2.4 

Case Management Activities, Year 2 




Individual Dosage 

Chrysalis students were grouped into high, medium, and low levels of participation in the program 
or “dosage" depending on how many sessions each student attended in the five major program 
components. A proposed “one-third model” was used to determine the level of dosage by program 
components (support groups, open sessions, challenge course, girls empowerment, and annual 
celebration). The optimal number of sessions for each component was divided by 1/3 to designate 
a high, medium, or low dosage level. Students attending 67 to 100 percent (top 1/3) of the available 
sessions in a component were identified as “high dosage.” Young women attending 34 to 66 percent 
(middle 1/3) of available component sessions were classified as “medium dosage” and youth 
attending one to 33 percent (lower 1/3) of sessions were grouped as “low dosage.” Students who 
did not participate in any sessions were classified as “no dosage” and eliminated from analyses. 
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For example, let’s look at the level of dosage in student support groups. During Year Two, a 
maximum of 24 student support groups were held in schools. Students who participated in 1 7 or 
more sessions were identified as the “high dosage” group. Young women attending 9 to 16 support 
group sessions were classified as “medium dosage” and those who participated in eight or fewer 
support groups were identified as “low dosage.” Youth who did not attend any support group 
sessions were classified as the “no dosage” group. More detailed information on these 
classifications is presented in Chapter 4 of this report. 

Analysis of dosage in Year Two indicates that 32 percent of Chrysalis students received “high 
dosage” by attending a total of 21 to 30 program sessions during 1995-96. Approximately 25 percent 
of students received “medium dosage” which is defined as participation in 11 to 20 sessions. 
Another 22 percent of students were classified as “low dosage” by attending one to 10 sessions. 
Twenty-one percent of students did not attend any project activity and were categorized as “no 
dosage.” 

Analysis of individual dosage for Year Two indicates that the average Chrysalis student: 

■ participated in 1 1 .3 support group sessions over 9.2 hours; 

■ attended approximately 3.9 of 7 Open Sessions during 3.1 hours; 

■ took part in 59 percent of the six-hour Girls Empowerment course; 

■ participated in 43 percent of the five-hour Challenge Course; 

■ attended 50 percent of the All-Student End-of-Year Celebration for four hours; 

■ received over 1 0 case management sessions over 3.9 hours; and 

■ participated in a total of 26.5 Chrysalis sessions over 31 .2 hours during the year. 
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Chapter 3. Evaluation Methodology 



Sampling Design / Approach 

The evaluation of the Chrysalis Project is a collaborative effort among the external evaluator, RMC 
Research Corporation, the internal evaluation team in the Portland Public Schools Research and 
Evaluation Department, and the project staff. This collaboration ensures that the information 
gathered through evaluation activities produces reliable evidence to assess program strengths and 
weaknesses, and is useful for improving program operations. Examples of this team approach 
include collaboration on instrument development, data collection activities for the validity study, and 
the case study of a small sample of students in the project. 

A variety of activities were conducted to gather information for the evaluation. Some activities were 
quantitative in nature, resulting in numeric descriptions of the program; other activities produced 
qualitative data, resulting in narrative descriptions which give more of a “real life” quality to the 
information. The findings presented in this report are a blend of both types of information. 

The impact of Chrysalis extends beyond the observable dimensions of reduced alcohol, tobacco, 
and other drug use, fewer health risk and victimization behaviors, and improved academic progress. 
The program affects different individuals in different ways and to a different extent. The intent of this 
evaluation is to gather evidence to answer the following research questions: 

■ What is the demographic profile of the students served by the project as measured 
by age/grade, gender, ethnicity, risk factors and type of abuse? 

■ To what extent do participants improve school performance as measured by 
improved attendance, reduced dropout rate, and progress in meeting graduation 
standards? 

■ To what extent do participants reduce negative consequences of child sexual 
abuse as measured by information on unsafe sexual practices and reduced early 
pregnancy, sexually transmitted diseases, and mental health disorders? 

■ To what extend do participants demonstrate improved resiliency as measured by 
resiliency scales on the Chrysalis Risk and Resiliency Index? 



Female students in grades nine, ten, and eleven who had been identified as victims of abuse were 
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contacted to participate in Chrysalis. Students who had participated in the pilot program were asked 
to sign a consent to “recommit” to participate in the program. Informed consent was obtained from 
the student, and with the student’s permission, the parent was then contacted. Both the student and 
parent were provided with an opportunity to ask questions and discuss the program prior to signing 
the consent forms. Student and parent consent were obtained from participants in the intervention 
and control groups. All participation is voluntary and students were permitted to withdraw from the 
program at any time. Once the consent forms were complete, students were randomly assigned to 
either the control group (termed Research) or the intervention group (termed Chrysalis Program). 

Research Design 

Chrysalis is a “true experiment” research design (Campbell & Stanley, 1961) featuring random 
assignment to Program and Research groups. This is the best approach for controlling rival 
explanations of effect, increasing internal validity, and strengthening causal inference (Campbell & 
Stanley, 1966; Mohr, 1992). The use of a control group with random assignment aids in accounting 
for potentially confounding influences such as history, maturation, testing, and regression. 

Exhibit 3.1 shows the number of students involved in the project each year. Exactly the same design 
will be used with the control group. The exhibit gives the actual number of participants for Year One 
and Two, along with recruitment goals for subsequent years. If these goals are realized, a total of 
1,810 intervention and control group youth will participate in the program over its four-year duration. 



Exhibit 3.1 
Evaluation Design 

Number of Participants in the Chrysalis Group by Year 





M 


iP 


1 




■ 






1994-95 


Pilot Cohort 
n=91 








1995-96 


Cohort 1 
n=189 


Pilot Cohort 
n=91 






1996-97 


Cohort 2 
n=240 


Cohort 1 
n=189 


Pilot Cohort 
n=91 




1997-98 


Cohort 3 
n=240 


Cohort 2 
n=240 


Cohort 1 
n=189 


Pilot Cohort 
n=91 


1998-99 


Cohort 4 
>n=240 


Cohort 3 
n=240 


Cohort 2 
n=240 


Cohort 1 
n=189 
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The Research group participates only in the evaluation activities and receives no special services 
that are not available to all district students, with the exception of the HIV/AIDS Education Project 
in Year Two. A randomized control group design ensures that characteristics of the control and 
intervention groups are similar at the outset of the project. In addition, information from the district 
population will often be included to serve as a baseline frame of reference for the target population. 

Case Study 

An in-depth case study of selected Chrysalis and Research students will augment the quantitative 
data with a qualitative study of the Chrysalis program. This intensive case study will focus on a 
sample of the high risk female adolescents in the project. The case studies will systematically 
describe the characteristics of the program population and provide comprehensive qualitative 
description, impressions, and judgements through a variety of interviews and observations. The 
case study will shed light on important variables, processes, and interactions that deserve more 
extensive attention in the evaluation. The scope of these case studies will encompass the entire 
span of student involvement starting with the initial student referral, through program participation, 
and following students longitudinally throughout the project’s funding cycle. The ongoing case study 
analysis will likely tell us things more quickly than the quantitative analysis and may offer meaningful 
information for formative evaluation. 

The purpose of the case studies is to explore how the Chrysalis project best supports its participants 
and explain what aspects of the project contribute to reduced risk and increased resiliency. Each 
case study should teach us more about the process and the need underlying it. The following 
themes provide a focus for the case study investigation. 

■ How important is Chrysalis compared with other life events in influencing whether and how 
students change? 

■ How important are the various components to the overall Chrysalis package? 

■ How engaged, committed, and honest are participants in their relationship with the project? 

■ What is the best target group for Chrysalis? Why are some students more successful than others? 

L 

In Year Two, case study students were selected to help study the relationship between the project 
and key risk characteristics of its participants. These characteristics included alcohol/drug use, 
violent behavior, child of a substance abuser, teen parent, and mental health issues. Cases were 
selected from different grade levels, ethnic groups, and risk factors to provide the most information. 
Exhibit 3.2 describes the demographic and risk factors of students in the Year Two case study. 
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Exhibit 3.2 

Case Study Sample, Year 2 















1 


Program 


17 


11 


African 

American 


Child of substance abuser, lesbian, and 
economically disadvantaged (3) 


2 


Program 


16 


10 


White 


Child of substance abuser, drug use, 
violent, juvenile court involved, witness 
to violence, knows someone murdered, 
mental health issues, foster home (8) 


3 


Research 


16 


10 


White 


Child of substance abuser, uses 
alcohol, tobacco & drugs, mental health 
problems, teen parent, and low socio- 
economic level (5) 


4 


Research 


18 


10 


White 


Child of substance abuser, dropout, 
school failure, pregnant, violent, mental 
health problems, chronic pain, suicide 
risk, and alcohol/drug use (9) 



Results of these qualitative data collection activities will serve many analytical purposes for the 
evaluation of the Chrysalis program. First and perhaps foremost, the case studies will provide the 
personal, human detail of the process and effects of the program on individual young women. As 
such, they will provide an invaluable supplement to the statistical results on the entire sample yielded 
by the quantitative analysis. The insights gained through the case study will have a dual effect on 
the quantitative analysis itself. First, they may. suggest specific analyses of factors unanticipated 
in the design. For example, personal interviews with students may suggest that certain aspects of 
their abuse history are particularly potent predictors of their current attitudes or health risk behaviors. 
These leads can be explored through correlational analyses on the full sample of young women, both 
Chrysalis participants and control. Second, the observations and experiences gained in personal 
contact with these young women will undoubtedly assist the interpretation of the quantitative 
analysis. It is highly likely that, given the wide variety of process and outcome variables included 
in this study, there will be patterns of relationships and moderating influences not immediately 
understood by the evaluation team. The qualitative portion of the study, along with discussions with 
program staff, is expected to contribute enormously to the interpretation of the results. 

Proposed and Actual Sample Size 

The project plans to recruit 480 students each year; 240 Chrysalis Program group students and 240 
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additional students in the Research control group. Year Two of the program was the first full school 
year implementation with students in cohort one. In Year Two, the actual number of project 
participants was 370 students; 189 Chrysalis young women and 181 Research control group youth. 

Properties and Characteristics of Operationaiized Dependent Measures 

Continuous assessment of student attitudes, achievements, and behaviors is paramount to the 
evaluation of this program. Therefore, we will involve all participants in no fewer than two evaluation 
activities each year. The evaluation will incorporate incentives to encourage students’ participation 
in the intake and data collection efforts (coupons, movie tickets, gift certificates, etc.). It should be 
noted that these instruments may change during the course of the project to continuously approach 
the state of the art in evaluation, respond to national cross-site evaluation directions from CSAP, and 
.reflect changes in education in Oregon. Copies of all instruments are in Appendix E of this report. 

Chrysalis Risk and Resiliency index 

The primary tool for assessing student outcomes is the Chrysalis Risk and Resiliency Index. It 
includes scales related to project activities and their impact on student behavior. The survey 
consists of scales from the Centers for Disease Control and Prevention’s Youth Risk Behavior 
Survey (YRBS), Phillips and Springer’s Individual Protective Factors Inventory (IPFI), and the CSAP 
Youth Survey being employed in the national cross-site evaluation of High Risk Youth Projects. 

The risk portion of the Chrysalis Risk and Resiliency Index includes a variety of health risk behaviors 
and attitudes from the YRBS and CSAP’s Youth Survey. These attitudes and behaviors include: 

■ Attitudes and perceptions of friends’ attitudes toward alcohol, tobacco, and other drugs; 

■ Use of alcohol, tobacco, and other drugs; 

■ Physical fighting and weapon carrying behavior; 

■ Sexual activity; and . 

■ Suicide ideation and attempts. 

Reliability analyses of YRBS data gathered on adolescents across the country are encouraging. 
Kappa coefficients from a test-retest administration of the survey to adolescents in grades seven 
through twelve ranged from .70 to .88 for the behaviors of interest to Chrysalis (Brener, Collins, 




Page 23 



35 



Project Chrysalis ■ Year 2 Evaluation Report 

Kann, Warren & Williams, 1 995). Further, these reliabilities were lowest for seventh and eighth 
graders, and higher for high school students. This lends great confidence to the use of this 
instrument among high-school-aged youth in the Chrysalis program. 

The resilience portion of the survey reflects intended outcomes of the project that relate to positive 
development. Children who succeed, despite having serious odds against them, are (a) more 
personally competent; (b) more socially competent; and c) more bonded to positive, pro-social 
influences (Benard, 1991). The Individual Protective Factors Inventory (IPFI), developed by Phillips 
and Springer (1992) operationalizes each of these three areas through three or four subscales. 

Personal Competence. The IPFI contains four subscales to represent the domain of personal 
competence, relating to one’s sense of personal identity. These subscales tap different dimensions 
of an individual’s ability to function effectively and make decisions that influence her future. 

■ Self Concept — a positive self-image or “feeling good” about yourself. This is a prominent 
constnjct in prevention literature, and may be thought of as one facet of a more global notion of 
self-esteem. 

■ Self Control — the ability to control impulses, particularly antisocial impulses such as anger or 
violence. This is a commonly employed moderator or outcome variable in the literature on 
predelinquent behavior or early risk behavior. 

■ Self Efficacy — the sense that life can have a purpose and that one’s actions can effectively 
achieve those purposes. This is a central theme in much of the literature on protective factors, 
similar to autonomy or internal locus of control. 

■ Positive Outlook — the general belief that life can have a positive outcomq and that it is 
attainable, even probable, for oneself. This relates to themes such as healthy expectations, 
success orientation, and achievement orientation in the literature on positive youth development. 

Social Competence. Three Subscales of the IPFI represent the construct of social competence, 
defined as the ability to be responsive, caring, and flexible in social situations. Anyone, youth or 
adult, having these abilities will likely elicit positive responses and reinforcement from others. This 
is a highly protective factor, relating to the formation of positive interpersonal relationships. 
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■ Assertiveness — the ability to stand up for oneself in social situations in reasonable ways. It is 
distinct from aggressiveness in that it connotes comfort, rather than hostility, in social situations. 

■ Confidence — The social dimension of self-esteem. It is the belief that one is liked and likable 
and that one will be accepted in a variety of social situations. 

■ Cooperation^ — the desire to contribute to social groups of which one is a part. It includes an 
internalized sense of accomplishment and satisfaction that comes with contributing. 

Social Bonding. The IPFI contains two subscales that assess the degree to which one has a 
positive affect and commitment to basic social institutions such as school and family. 

■ School Bonding — positive affect and motivation towards school, both currently and in the future. 
It includes the belief that education is important to future success and personal accomplishment. 

■ Family Bonding — positive affect toward family: a perception of support and positive interaction. 

Support/Guidance. The IPFI contains three subscales to represent support/guidance, and relating 
to one’s sense that there is a person who can be trusted when help or advice is needed. 

■ Nurturance — the feeling that there is someone who cares and can be depended on when help 
is needed. 

■ Guidance — the sense that there is a person in one’s life who can be trusted to talk about 
problems with and be called upon for advice. 

■ Support — this is a combination of nurturance and guidance. It is the belief that there is someone 
who can be depended on for help when necessary and will be available for trustworthy advice. 

Psychological constructs such as these are often difficult to measure. The authors of the IPFI 
compiled substantial psychometric data on the instrument in their prior use of it on a national sample 
of students, ranging from fifth to twelfth graders, and including all racial/ethnic groups. To justify the 



Vhis scale was deliberately revised by the evaluators for this project. Project staff felt that the typical definition of cooperation 
and teamwork was not the goal for this population. These are young women who may have cooperated too much or been influenced too ^ 
readily by abusive peers or adults. Reciprocity and making go^ choices in working with others are the key aspects of cooperation here. 




Page 25 

37 



Project Chrysalis ■ Year 2 Evaluation Report 



use of the IPFI as a critical outcome instrument in this study population, the authors calculated 
internal consistency reliabilities on all subscales. These reliability data, from both the field test 
conducted by the instrument’s authors (Phillips & Springer, 1 992) and the current Chrysalis Project, 
lend considerable confidence in the reliability of the instrument. The coefficients in Exhibit 3.3 are 
evidence of excellent reliability even at the subscale level, most of which consist of only six items. 



Exhibit 3.3 

Reliability Coefficients of IPFI, Year Two 

from National Field Test and Current Application 





Personal Competence 12 


Self Concept 


3 


.58 


.44 


Sell Control 


3 


.65 


.67 


Self Efficacy 


3 


.56 


.37 


Positive Outlook 


3 


.56 


.70 


Social Competence 1 0 


Assertiveness 


3 


.46 


.60 


Confidence 


3 


.59 


.69 


Cooperation 


4 


.65 


.71 


Social Bonding 6 


School 


3 


.61 


.67 


Family 


3 


.58 


.81 


Support/Guidance 1 0 


Nurturance 


3 


NA 


.61 


Guidance 


3 


NA 


.79 


Support 


4 


NA 


.78 



Other Assessment Instruments 

Other data collection forms and evaluation instruments are used to document the implementation 
and effectiveness of the project, gather information on individual needs and priorities for program 
services, student achievement, attendance, dropout rates, and progress toward meeting the district’s 
graduation standards. Descriptions of these tools and the frequency of their use follows. 
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Chrysalis Intake Form. This form collects information on students referred to the program and 
facilitates the random assignment of girls to program or control groups. A scannable intake form was 
developed for use in Year Three. (The form is completed once, as the student enters the program.) 

Needs Assessment This instrument collects information oh students selected to participate in the 
program. The assessment asks students to identify their interests and needs and helps case 
managers establish priorities for service. (The form is completed once for each program student.) 

Chrysalis Follow-up Survey. This abbreviated form of the Risk and Resiliency Index focuses on 
key subscale items to assess the longitudinal impact of the project after students have received two 
years of services. (The survey will be administered yearly to participants in post years 1 and. 2.) 

Chrysalis Database. The Chrysalis database tracks information on intervention and control group 
students. It consolidates demographic and program participation information from existing sources, 
such as the PPS Student Master Database, i.e., gender, ethnicity, test scores, grades, absences, 
and disciplinary violations. The database contains information such as the following: 

■ Graduations Standards Test/CARAT. The Graduation Standards Tests (GST) in Reading and 
Mathematics must be successfully passed for a student to obtain a standard diploma. 
Students may complete either the paper-and-pencil test or the electronic Computerized 
Adaptive Reporting and Testing (CARAT) version of the Graduation Standards Test. (These, 
tests will be administered yearly to students in the program who have not yet passed.) 

■ School Grades — Student grade point averages are recorded from school records. (GPA is 
calculated once each year for all students involved in the program.) 

■ Number of Absences — ^The total number of annual student absences is recorded from the 
District database. (Information is collected in June.) 

■ Number of Suspensions — The number of suspensions students had each year is recorded 
from the District database. (Information is collected in August.) 

Case Manager’s Contact Log. This log documents ongoing case management contacts, activities, 
and communication with project students. (This form, is submitted monthly by case managers.) 
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Chrysalis Activity Log. This log is also filled out by case managers to identify activity topics and 
to document students’ participation in all Chrysalis activities. Data from these logs are used to 
quantify the dosage level of the program for each student. 

Girls’ Empowerment Pre-Post Survey. This brief survey is administered to the Chrysalis 
participants before and after their participation in the Girls Empo\werment Course. It assesses their 
level of knov/ledge, attitudes, and skills in a variety of physical safety and self-defense areas. 

Methods / Procedures 

Evaluation data were collected directly from Chrysalis and Research group students, from school 
records, and from project staff. Substantial information on students was collected from existing data 
files to minimize intrusion on students and staff. For example, the district’s student master database 
provides the capability to follow students who move from school to school during a school year and 
from grade to grade over the years of the project. These longitudinal files and information systems 
provide abundant baseline data to document the extent to which the project increased prevention 
capabilities among the young women in the project. It is important to note that the student database 
is operated in ways that ensure the confidentiality of student data (online access is secured by 
multiple passwords, access-protected files, and data are stored in locked offices). 

Exhibit 3.4 summarizes the data collection instruments and timelines employed in Year Two. All 
information is summarized and reported annually to project staff and school-based case managers. 
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Exhibit 3.4 

Summary of Data Collection Procedures, Year Two 







Chrysalis Intake Form 


Student Intake 


Sep. 4, 1995 - Dec. 15, 1995 


Process Evaluation 


Needs Assessment 


Program planning 


March 1996 


Case Manager Contact Log 


Documents contacts with 
students by case manager 


Monthly 


Attendance/Activity Log 


Documents attendance and 
curriculum dosage 


January and June 1996 


Girls Empowerment Survey 


Assessment of physical 
safety & self-defense skills 


Winter 1 996 


Challenge Course Survey 


Measure of locus of control 


Spring 1996 


Program Observations 


Description of project 
activities 


Quarterly 


Interviews 


Assessment of satisfaction 
with program 


Quarterly 


Case Studies 


Student stories 


Bi-monthly 


Outcome Evaluation 


Risk & Resiliency Index 


Assessment of risk and 
resiliency factors 


October 1995 (pretest) 
May 1 996 (posttest) 


CSAP National Youth Survey 


Student attitudes & drug use 


Planning for Year 3 implementation 


School Indicators: 

Graduation Standards Test 
School Grades 
School Attendance 
School Discipline 


Progress towards 
graduation standards, 
academic achievement, 
school performance, and 
school bonding 


April 1 996 
June 1996 
July 1996 
August 1 996 


Interviews 


Outcomes and impact of 
project 


Quarterly 



T raining of Data Collectors 

All members of the Project Chrysalis research team that have contact with project students have 
been trained in the current literature related to high-risk youth, specifically as it relates to female 
adolescents and physical/sexual abuse. Observations and case study research are conducted by 
female researchers trained in cultural competence and sexual abuse issues. The Waverly contract 
child abuse therapist provided a four-hour advanced training in the trauma model for program staff 
and the evaluation team in January 1996. 
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Privacy and confidentiality is of the highest priority in the evaluation of this project. The members 
of the evaluation team received orientation and training in confidentiality at the beginning of the 
project and protocols were established for transferring information, contacting students, and other 
necessary data collection procedures. Data collection which requires student contact was done only 
by evaluators trained in sexual abuse issues. Access to student data is strictly on a “need to know" 
basis with only the Chrysalis coordinator, school case managers working with given students, and 
the evaluators having access to student information. 

State laws and district regulations require staff and evaluators to report abuse regardless of a 
student’s wish for confidentiality. Students are informed at their first contact with the project, prior 
to intake, that staff must comply with child abuse reporting procedures. All incidents of abuse are 
verified with the state child protection services agency. Students who reveal physical, sexual, or 
emotional abuse to any project staff are informed that a report of the incident will be made to school 
police. At this time, students edso receive counseling support and referral to appropriate professional 
treatment providers in the community. Chrysalis maintains confidentiality of client records in 
accordance with the provisions of Title 42 of the Federal Regulation Code, Part 2 (42 CFR, Part 2). 

Data Analysis 

Data collection and analysis activities in the Year Two evaluation of the Chrysalis project included 
both qualitative and quantitative efforts. Qualitative data collection and analysis activities consisted 
of observations, interviews, and case studies of a small sample of Chrysalis and Research group 
participants. Quantitative analysis of self-report survey (Risk and Resiliency tnde)(i and student 
records (Portland Public Schools Student Master Database) data consisted of the following methods: 

Descriptive statistics were computed to represent the youths’ prevalence rates on key risk 
behaviors, average scale scores on protective factors and indicators of academic performance. 

Correlations were computed to assess the relationship between levels of program participation 
(dosage), initial risk level and protective factors, health risk behaviors and academic indicators. 

Inferential analyses — chi-square and analysis of variance (ANOVA) — were conducted to 
determine \A^ether relationships among the process and outcome variables, or changes over 
time, were statistically significant. 
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Results presented in the following sections and chapters of this report will be those obtained from 
various combinations of these methods. For example, in looking at the effects of the Chrysalis 
program on the sexual behavior of these young women, the prevalence rates of sexual intercourse 
and pregnancy will be displayed in a descriptive manner for both Chrysalis and Research groups; 
correlations between these behaviors and the risk level of all participants will be reported; 
correlations of these behaviors with program participation (total dosage and by program component) 
will be presented for Chrysalis participants; and the significance of any changes in these behaviors 
will be assessed though a multi-factor ANOVA design. 

The Repeated Measures ANOVA design used to assess the significance of change is shown in 
Exhibit 3.5 below. The design consists of two "between subjects" factors by which students are 
uniquely classified: Group (Chrysalis or Research) and Grade Level (9, 10, 11 or 12). It also 
includes a "within subjects" factor that crosses all students: Time (fall pretest and spring posttest). 



Exhibit 3.5 

Repeated Measures ANOVA Design 











Chrysalis Group 


Grade 9 






Grade 10 






Grade 1 1 






Grade 12 






Research Group 


Grade 9 






Grade 10 






Grade 1 1 






Grade 12 







The design includes seven testable effects: three main effects, three first order (two-way) 
interactions, and a single second order (three-way) interaction. Significant design effects will be 
reported in the discussion of results for each of the key outcomes. To assist in the interpretation of 
these effects, brief definitions that apply to all outcomes are provide on the following pages. 
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Main Effects 

The Group main effect assesses whether the difference between Chrysalis and Research 
group students (across time and grade levels) is significant. 

The Grade Level main effect assesses whether the differences among students at different 
grades, regardless of what group they are in and across the two time periods, is significant. 

The Time main effect assesses whether the difference across the two time periods (Fall and 
Spring of this Year Two of the project) is significant, regardless of which group or grade level 
the young women are in. 

First Order Interactions 

The Group by Grade interaction assesses whether any differences between the two groups 
are the same or different across grade levels. It essentially tests whether the results of the 
Group main effect apply consistently to all grade levels, or whether it is different depending 
upon which grade level a student is in. 

The Group by Time interaction assesses whether changes over time are consistent for both 
groups of students. 

The Grade by Time interaction assesses whether changes over time are consistent for 
students in each grade level. 

Second Order Interactions 

The Group by Grade by Time interaction assesses whether any of the first order interactions 
described above are consistent for all levels of the third factor. For example, if a significant 
Group by Time interaction were found (e.g., that Chrysalis students showed an increase in 
a given outcome, while Research students did not), but that this was the case only for ninth 
and tenth graders, and not for eleventh and twelfth graders, this would represent a Group by 
Grade by Time interaction. . 
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In the inferential analyses discussed throughout the Results section, the effects of primary interest 
will be all of those involving the Group design factor, i.e., the Group main effect, the Group by Time 
interaction, the Group by Grade interaction and the Group by Grade by Time interaction. All of these 
include the fundamental comparison of interest to the study — the difference between young women 
in the Chrysalis and Research groups. 

A second set of Repeated Measures ANOVAs was run, using a design that is parallel to that shown 
earlier in Exhibit 3.5. In these analyses, however, a Risk Leve/ design factor was substituted for the 
Grade Level factor in the design shown above. All eligible students. Chrysalis and Research, were 
classified as high-, medium- or low-risk based on the number of risk factors elicited at their Intake 
interview. Low-risk students were those having one to three risk factors (note that all students in 
both groups had, by definition of their eligibility for the project, at least three risk factors: victims of 
^use and two other risk factors); medium-risk students possessed four to seven risk factors; and 
high-risk students reported eight or more risk characteristics. 

Results from both sets of ANOVAs will be described throughout the following chapter. 
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Chapter 4. Results 

Analysis of Initial Comparability of Treatment and Comparison Groups 

The Project Chrysalis evaluation used a random assignment technique to ensure that the students 
selected for the intervention or Program group and the control or Research group had similar 
characteristics at the start of the project. During Year Two of the project, random assignments of 
students to the intervention and control groups were based primarily on the demographic 
characteristics of age, grade, and ethnicity. Multiple analyses in Year Two confirmed the initial 
comparability of the intervention and control groups in demographic characteristics and risk factors. 

Demographic Characteristics 

The age, grade level, and geocultural group distributions of the 1 89 Chrysalis Program (intervention) 
students and the 181 Research group (control) students in Year Two are shown in Exhibits 4.1 , 4.2, 
and 4.3. Statistical significance tests, using the chi-square technique, yielded no significant 
differences between these two groups on all three indicators. Exhibit 4.1 shows that the age of the 
average Project Chrysalis student in Year Two was 1572 years. 



Exhibit 4.1 

Age of Female Adolescents, Year Two 




12-13y 14y 15y 16y 17y 18y 19y 20y 

Age 
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Exhibit 4.2 displays information on the grade levels of participants during the 1995-96 school year. 
Most students in Year Two were freshmen; approximately 39 percent of the participants were in 
grade nine. Students in grade 10 and grade 1 1 made up approximately 25 percent of the groups, 
with a small number of students from other grades. 



Exhibit 4.2 

Grade Level of Female Adolescents, Year Two 
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Exhibit 4.3 shows the composition of the Program and Research group students by geocultural 
group. The Chrysalis and Research groups are virtually identical in their geocultural composition. 
Another frame of reference in the examination of ethnicity is the composition of geocultural groups 
in the full high school populations of the twelve participating schools in the district. The overall 
project has a slightly higher proportion of American Indian students and a lower proportion of Asian 
American students than the general population represented in the district high schools. The ethnic 
enrollment in the district high schools was: 1.8 percent American Indian, 66.5 percent European 
American, 16.1 percent African American, 11.0 percent Asian American, and 4.6 percent Hispanic 
American. In Year Three, the project continues to investigate further strategies for improving the 
participation of underrepresented groups. In addition, the random assignment process was adjusted 
to select more minority students into the intervention, when other variables are equivalent. 
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Exhibit 4.3 

Geocultural Group of Female Adolescents, Year Two 



100 






Program (n= 189) 





Research (n=181) 
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Geocultural Group 



Risk Factors 

During the second year of the project, students were referred for possible enrollment based on an 
array of 24 risk factors. The intake interview, which preceded the random assignment to groups, 
systematically assessed whether or not each of these risk factors was in evidence in each student’s 
background. Exhibit 4.4 summarizes the top 12 risk factors common among Chrysalis Program and 
Research group students in Year Two. The risk factors were also categorized by domain: school, 
individual, family, peer, and community. Chrysalis and Research group students primarily show risk 
indicators in the individual, family, and school domains. The most frequently reported risk factors 
in the individual domain were childhood abuse and severe emotional problems. The most frequently 
reported risk factor associated with the family domain was a family history of substance abuse. The 
school domain’s most common risk factor was dropout/chronic school failure. The types and 
prevalence of risk factors reported for students at intake were consistent over year one and two. 

As is clearly shown in the display, all students had a history of physical or sexual abuse — this was 
the defining characteristic of the target population of the project. Nearly two of every three students 
indicated they were children of substance abusers or reported severe mental health problems. Over 
half were experiencing chronic school failure, suicide ideation, or economic disadvantages. 
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Exhibit 4.4 

Prevalence of Top 12 Risk Factors, Year Two 



Victim Of abuse 
Severe emotional problems 
COSA 

School dropout/failure 
Attempted suicide 
Economically disadvantaged 
Witness to violence 
Runaway 
Violent/delinquent 
AOD use 
Domestic violence 
Pregnant 
Tobacco use 




40 60 

Percent of Students 



Project Chrysaiis modified the student intake form to expand the number of risk factor characteristics 
in August 1995. Based on current prevention iiterature, the iist of risk factors was expanded to 
inciude other variabies, such as tobacco use, witness to vioience, victim of domestic or schooi 
vioience, runaway, and parent invoivement in the criminai justice system. As in the demographic 
comparisons, there were no significant differences between the two groups of students on the 
prevalence of any of these risk factors. This provides great confidence that the two groups can be 
considered equivalent on these baseline indicators. A complete summary of the Year Two risk factor 
characteristics for the Program and the Research group students is in Appendix E. 

Exhibit 4.5 summarizes the type of abuse reported by Chrysalis students at intake. Abuse 
categories are physical, sexual, emotional abuse, or any combination. A third of the participants 
have experienced emotional abuse. One in four students has a history of sexual abuse. 
Approximately 10 percent of the students have been physically abused or had some combination 
of physical, sexual, and emotional abuse. The abuse history of the Chrysalis Program group and 
Research group students is deemed equivalent across the various types of abuse. 
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Exhibit 4.5 

Student History by Type of Abuse 




Sexual Physical Sex/Emot 

Emotional Phya/SexI Phys/Sex/Emot Phys/Emot 

Type of Abuse 



Attrition / Attrition Analysis 

As was noted in Chapter 2 of this report, the project benefited from having 91 Program girls who had 
participated in the year one pilot, continue their participation during Year Two. This resulted in less 
attrition among the continuing students as only seven Chrysalis youth did not recommit to participate 
in the Year Two program. Chrysalis has defined three levels of program participation and attrition: 

1) active students are enrolled in school and participating in both program and evaluation activities, 

2) level 1 inactive students are “inactive” in the project either because they are not enrolled in school 
or did not participate in at least one annual outcome evaluation assessment, and 3) level 2 inactive 
students are not enrolled in school and not participating in any evaluation activities. 

Of the 189 Chrysalis Program youth, 67 percent (n=126) were active at the end of Year Two. The 
Program group lost 63 students to attrition; 24 percent (n=45) were level one inactive, youth not 
involved in the program, but active in evaluation activities and 9.5 percent of the girls (n=18) were 
level two inactive in both program and evaluation activities. Among the 1 81 Research youth, 70 
percent (n=127) were active at the end of the year. Attrition in the Research group was 54 students; 
24 percent of the girls (n=44) were level one inactive either due to dropping out of school or not 
completing at least one outcome evaluation during the year. Six percent of the Research girls (n=1 0) 
were level two inactive 6oe to not being enrolled in school and not participating in the evaluation. 
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These data slightly overestimate attrition as some students completed pre and post-surveys through 
first-class mail and thus remained actively involved in the evaluation activities. Data on student 
participation indicated that of the 63 “inactive” Chrysalis students in Year Two, 22 were school drop- 
outs, 14 moved out of the district, 1 1 transferred to GED programs, 7 did not recommit to take part 
in Year Two activities, 5 were runaways, and 4 were untraceable after three attempts to reach them. 

Analysis finds that the overall attrition rate for the Program Group is 9.5 percent (n=18) vs. a 5.5 
percent attrition rate for the Research Group (n=10). The project staff speculate that this slightly 
higher rate of attrition among Program participants is due to the difficult self-work that the young 
women are asked to do in Chrysalis support groups to recover from child physical, sexual, and 
emotional abuse. Attrition is lower among Research students because they are not participating in 
recovery-related program activities and because they value the incentives they receive for 
participation in the evaluation activities. The demographic characteristics of the “active students” 
analyzed in this report are equivalent to the whole group of Chrysalis participants at intake. 

Statistical Analysis for Key Dependent Measures 

The intended outcomes of Chrysalis fall into several distinct areas, as previously noted. Before 
examining the results of those comparisons, however, a word about the limitations of the data is in 
order. The early years of any demonstration project are fraught with the difficulties of initiating an 
innovative program: changes in staff, recruiting new students, developing program fidelity, 
establishing reliable data collection, etc. Chrysalis was no stranger to these difficulties. While the 
project refined staff and implementation issues, the data collection was also restructured. During 
Year Two, the evaluators administered the pre and post Risk and Resiliency Index \o students at the 
students’ schools or via first-class mail to their homes. While this maximized the number of surveys 
that could be collected, it also increased the potential for error. A third of the project participants 
completed only one outcome assessment, either because they were absent from school the day the 
survey was administered or because they did not return the mailed survey. This eliminated 
significant data from the analysis. The statistical analysis was limited to only students with complete 
Risk and Resiliency Index data (fall pretest and spring posttest) in order to measure change in the 
attitudes or behaviors of the young women. The number of pre and post Risk and Resiliency 
surveys analyzed for Year Two was 76 for the Program Group and 96 for the Research Group. 

The analysis of four sets of dependent measures, along with two case studies, will be presented: 
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• the initial needs assessment for Chrysalis students; 

• results for Chrysalis and Research groups for the health risk behaviors measured by the Risk 
and Resiliency Index; 

■ results for both groups for the protective factor portion of the Risk and Resiliency Index; and 

• results for both groups in the analysis of school achievement and attendance records. 

Needs Assessment 

Case managers interviewed each of the Program students at the outset of the project to determine 
v\rhich specific areas of their lives the young women were most interested in improving. These data 
assisted case managers in tailoring the work with the young women to correspond with their own 
stated needs. At an aggregate level, these data assisted project management in determining the 
extent to which the program design was congruent with the expressed needs of the participants. 

A needs assessment survey was distributed to new Year Two participants (n=53) in March 1996. 
Exhibit 4.6 lists the possible areas the girls could choose, along with the percentage of students 
selecting each area as one of the three they were most interested in. The tendency to choose 
concrete, material needs is evident in these results. The most frequently selected areas in Year 
Two needs assessment were money to buy something for myself, transportation, and getting a job. 



Exhibit 4.6 

Needs Assessment Summary, Year 1 and Year 2 

Mv life would be better if I got some help on... 









Money to buy for myself 
Transportation 
Getting a Job 

Schoolwork ^ 





Number of Students 
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Health Risk Behaviors 
Alcohol Use 

The prevalence of varying patterns of alcohol use for both Chrysalis and Research group students 
is shown in Exhibit 4.7a. The data represent students who responded to these questions at both the 
pre and post assessment points (fall 1995 and spring 1996) in order to assess students’ level of use 
over time for each group. They also include only students who received a sufficient "dosage" of 
Chrysalis activities deemed by project staff to be one-third of all available program activities. 



Exhibit 4.7a 

Prevalence of Alcohol Use 

Year 2 Fall and Spring Results for Chrysalis and Research Groups 




Chrysalis Research Chrysalis Research 

SO-Day PrevalerKe 3 or More Drills in Past 30 Days 



Based on students taking both Fall and Spring surveys and 
(Chrysalis) participating in at least 1/3 of the program activities 
(Chrysalis N-76; Research N»96) 



At the beginning of the school year, about four out of ten Chrysalis and Research girls said they had 
been drinking in the past 30 days. In the spring, this number essentially remained the same for 
Research students, while it increased to one out of two Chrysalis students who said they drank in 
the past month. 
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The percentage of both Chrysalis and Research students who reported consuming three or more 
drinks on the days that they drank was reduced slightly between pre and posttest in Year Two. The 
rate of Chrysalis students declined from 25.3 percent in the fall to 22.7 percent in the spring, and the 
percentage of Research students declined from 27.7 percent to 25.5 percent. 

The primary evaluation interest is in the difference between Chrysalis students and Research 
students over time. A repeated measures analysis of variance (ANOVA) is the design used to test 
the statistical significance of differences in this study. As described earlier, the two repeated 
measures designs control for grade level, risk level, and time, as the prevalence of most health risk 
behaviors tends to increase with grade and risk level. The ANOVA results for the Group x Risk x 
Time design are shown in Exhibit 4.7b below. The factor of most interest is the two-way Interaction 
between "Group and Time." A significance level at or below 0.05 on this factor indicates that 
Chrysalis and Research students have changed in significantly different ways between the fall and 
spring assessment points. 



Exhibit 4.7b 

Repeated Measures ANOVA for Alcohol Use 





Use of Alcohol in Past 30 Days 


Number of Drinks When Drank 
Alcohol in Past 30 Days 












Group 


.18 


.671 


.08 


.772 


Risk Level 


.95 


.388 


1.14 


.322 


Time 


1.93 


.167 


2.51 


.115 






Group X Risk 


1.24 


.292 


.37 


.691 


Group X Time 


3.56 


.041* 


.03 


.865 


Risk X Time 


.76 


.469 


.11 


.897 










1 Group X Risk x Time 


.90 


.407 


.02 


.977 



* p<.05 



The Two-Way Interaction between Group and Time on 30-day alcohol use is statistically significant 
(p<05), meaning that it is likely that the difference in the number of days students drank in the past 
30 days between the two groups over time is different, depending upon the group to which a student 
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belonged. In other words, Chrysalis students changed their prevalence of alcohol consumption in 
the past month in ways different than Research students. Unfortunately, this change is not in the 
direction hoped. The percentage of Chrysalis students reporting consuming alcohol in the past 30 
days increased by 12 percent from the beginning of the school year to the end, while Research 
group students increased by about 2 percent. When they drank alcohol in the past month, both 
Chrysalis and Research students drank similar amounts, as there are no significant differences in 
the repeated measures ANOVA. 

Among the Chrysalis girls only, there was a significant correlation between their participation in the 
Girls Empowerment program component and 30-day alcohol use. Fewer of the Chrysalis young 
women who participated in this component reported alcohol use in the previous 30 days. This 
suggests that this program component has a positive influence on alcohol use among these young 
women, and has implications for program staff in the coming year. While these correlational results 
never prove causality, it clearly merits the recommendation to encourage greater participation in Girls 
Empowerment in the coming year. 

Unlike most of the other health risk behaviors reported here, there was no relationship between 
alcohol use and students' entering risk level in either group. It may be that alcohol use in this 
population and among high school students in general is a common enough adolescent health 
behavior that risk background does not explain it. Grade level differences (not shown in Exhibit 4.7b) 
were similarly non-significant, although there were small differences to indicate that older students 
were more likely to drink than younger students. 

Cigarette Smoking 

The self-reported prevalence of cigarette smoking is displayed in Exhibit 4.8a. Tests of significance 
conducted through the repeated measures ANOVA follow in Exhibit 4.8b. More than half of all 
project students at both assessment points said they have smoked cigarettes in the past 30 days. 
The percentage of Research students who answered ”yes” to this question remained essentially the 
same between fall (58.8%) and spring (57.7%). Chrysalis students increased theirlise. In the fall, 
they were about seven percentage points below Research students. Their rate increased by about 
9 percent, putting the percentage of Chrysalis girls slightly higher than Research youth in the spring. 
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Exhibit 4.8a 

Prevalence of Smoking Cigarettes 

Year 2 Fall and Spring Results for Chrysalis and Research Groups 




Chrysalis Research Chrysalis Research 



30-Day Prevalence Smoking 2 or More 

Cigarettes Per Day 

Based on students taking both Fall and Spring surveys and 
(Chrysalis) participating in at least 1/3 of the program activities 
(Chrysalis N-76; Research N=96) 



Exhibit 4.8b 

Repeated Measures ANOVA for Cigarette Use 





Use of Cigarettes in 
Past 30 Days 


Number of Cigarettes Smoked 
When Smoked in Past 30 Days 




//v., /IjALv 






gfSiinWic^oe:;;. 

-^ 5 '- *^“teve! " " ' 


Group 


O 

CD 


.763 


.63 


.429 


Risk Level 


2.91 


.057 


1.25 


.289 


Time 


16.99 


.001* 


.19 


.663 




Group X Risk 


1.04 


.357 


1.42 


.244 


Group X Time 


3.39 


.067 


.14 


.708 


Risk X Time 


1.86 


.160 


CVJ 

p 


.978 




Group X Risk x Time 


2.10 


.126 


.72 


.490 



* p<.05 
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When taking the full pattern of number of days smoking into account in the ANOVA, there is a highly 
significant Time main effect (p<.001) and a marginally significant Group by Time interaction. Looking 
in greater detail at responses to these items, about one-third (32%) of the young women in both 
groups smoked on 20 or more of the past 30 days in the fall, while this rate rose to over 41 percent 
in the spring across both groups. So, while the percent of students who smoked at all rose more in 
Chrysalis than among the Research group, the actual frequency of smoking (the number of days 
smoking rather than simply whether or not they smoked) was more similar among both groups over 
time. This significant increase in the frequency of smoking for both groups, the Time main effect, 
is a serious concern for this population, however, and one the project may want to address directly. 

On the days they smoked, both Chrysalis and Research students increased the number of cigarettes 
they smoked. Similar to the prevalence of cigarette smoking, the percentage of Chrysalis students 
was lower than Research students in the fall, but increased to essentially the same level as 
Research students in the spring. These differences were not statistically significant, however. 

None of the program participation (dosage) indicators related to cigarette smoking among Chrysalis 

students were significant. Initial risk level was moderately related to this behavior, as evidenced by 

/ 

the marginally significant Risk Level main effect (p<.057) in the ANOVA table. This relationship- 
indicated that the higher risk students in both groups were more likely to have higher prevalence 
rates of cigarette smoking. There were also no significant differences across grade levels. 

Marijuana Use 

The self-reported prevalence of marijuana use is illustrated in Exhibit 4.9a. Tests of significance 
conducted through the repeated measures ANOVA follow in Exhibit 4.9b. In the fall, a slightly higher 
percentage of Research students reported using marijuana in the past 30 days than did Chrysalis 
students. In the spring, however, a far greater proportion of students said “yes” to this question, 
while the percentage of Research students essentially stayed the same. 

A similar pattern emerges when students were asked to report the number of times they used 
marijuana on the days that they used it. In the fall. Chrysalis and Research students reported similar 
use of marijuana in each category: once a day, twice a day, and three or more times a day. In the 
spring, the percentages for Research students decreased slightly, while for Chrysalis students the 
frequency of use increased. 
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Exhibit 4.9a 

Prevalence of Marijuana Use 

Year 2 Fall and Spring Results for Chrysalis and Research Groups 




0> 

“O 



CO 



<D 

o 

0> 
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Chrysalis Research Chrysalis Research 



30-Day Prevalence Use of Marijuana More Than Once 

a Day When Used in Past 30 Days 

Based on students taking both Fall and Spring surveys and 
(Chrysalis) participating in at least 1/3 of the program activities 
(Chrysalis No76; Research N°96) 



I 




Exhibit 4.9b 

Repeated Measures ANOVA for Marijuana Use 





Use of Marijuana 
in Past 30 Days 


Amount of Marijuana Used 
When Used in Past 30 Days 


Main Effects, 1, 


V " " X" *' ' ' 


''’fS^nlficance'':v 

' ' " ' Level ' 




Significance, 1 


Group 


2.59 


.109 


1.18 


.279 


Risk Level 


3.31 


.039* 


4.69 


.010* 


Time 


4.46 


.036* 


1.60 


.208 


Two-Way Interaction Jell'S ^ S' 'L.,' ' S 


Group X Risk 


10.32 


.019* 


3.01 


.042* 


Group X Time 


4.46 


.036* 


6.21 


.014* ‘ 


Risk X Time 


1.49 


.229 


.83 


.436 


Thre^Way. Interaction -’'r- ” ^ 


Group X Risk x Time 


1.49 


.229 


.49 


.612 



■ P<.U5 
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The ANOVA results in Exhibit 4.9b confirm that changes over time in use of marijuana in the past 
30 days between the two groups are statistically significant (Group x Time interaction, p<.036). There 
are also significant differences among students at different risk levels (p<.039), and a significant 
Group X Risk interaction. In general, students at higher risk levels were more likely to be marijuana 
users. The Group x Risk interaction somewhat modifies the generalizability of this relationship. 

The Group by Risk interaction is particularly interesting in that it suggests that increases in marijuana 
use associated with increasing risk levels among the students are not the same in Chrysalis and 
Research groups. Exhibit 4.10 displays the pattern of 30-day marijuana use for high, medium and 
low risk students in both groups at posttest time. In the Research group, there is increased 
marijuana use among "moderate risk" students over those at "low risk," but there is little change in 
marijuana use between "moderate risk" and "high risk" students. In Chrysalis, however, the lowest 
level of marijuana use is among "moderate risk" students, suggesting perhaps that the program is 
most successful in reducing marijuana use among students at this risk level. In viewing these 
results, program staff are discussing the extent to which their activities are tailored more to this 
group than those in either low or high risk categories. 

Results on the amount of marijuana used in the past month by the two groups presents a similar 
picture. The two-way Interactions of Group by Risk and Group by Time are statistically significant 
(p<.042, p<.014, respectively), indicating that it is highly likely that the difference between the two 
groups in the amount of marijuana used over time can be explained by the group and risk level to 
which they belong. When they used marijuana. Chrysalis students used more at the end of the 
school year than they did in fall, while Research girls slightly decreased the amount they used. 

J 

Higher levels of program participation again influence these behaviors among Chrysalis students. 
A significant negative correlation exists between the number of support groups attended and 30-day 
marijuana use, meaning the more support groups a Chrysalis student attended, the lower her 
reported use of marijuana. Similarly, participation in the Girls Empowerment component is 
associated with lower marijuana use rates. Implications for the Chrysalis program are again clear: 
program staff \will focus even more strongly on increasing participation in the support group and Girls 
Empowerment components in an effort to more positively affect this troubling behavior. 
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Exhibit 4.10 

Relationship Between Risk Level and Marijuana Use 
for Chrysalis and Research Group Students 




6-9 Days 



No. of Days Used 
Marijuana in Last Month 



3-5 Days 



1-2 Days 



3.24 




♦Chrysalis 

♦Research 



None 



Low Risk Moderate Risk High Risk 

Level of Risk 

Prevalence of ^xual Activity 

Exhibit 4.11a presents information on the prevalence of sexual activity among Chrysalis and 
Research Group students on the pre and post assessment. The percentage of students who 
reported being sexually active in both groups is very similar. In the fall, 78.7 percent of Chrysalis 
students answered “yes” to this question, and in the spring it was 84.0 percent. For Research 
students these percentages were 79.4 (fall) and 84.5 (spring). This approximately 5 percent 
increase in both groups may indicate the percentage of students having intercourse for the first time 
during this period, or it could reflect unreliability in their responses to the question. In either case, 
this level of lifetime prevalence in both groups is far higher than that reported by high school females 
across the nation (52%: CDC, 1996). 
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Exhibit 4.11a 

Prevalence of Sexual Activity 

Year 2 Fall and Spring Results for Chrysalis and Research Groups 

100 



80 



60 



40 



20 



0 

Chrysalis Research Chrysalis Research Chrysalis Research Chrysalis Research 

Lifetime Prevalence Age of Onset 4 or More Sexual 4 or More Sexual Partners 

1 2 Years Old or Younger Partners in Lifetime in Past 3 Months 



■Fall ’95 Hspring ‘96 



84 84.5 




Based on students taking both Fall and Spring surveys and 
(Chrysalis) participating in at least 1/3 of the program activities 
(Chrysalis N=76; Research N=96) 



In the spring survey, more Research students (20.7%) than Chrysalis students (14.7%) reported 
having sexual activity at or before age 12. The four percent increase among Chrysalis students from 
fall to spring may reflect increased willingness to disclose this personal information about their past. 
Again, these rates are far higher than the national average. 

An additional survey item added in the spring asked these young women whether their first sexual 
intercourse was an "abusive incident." More than half (54.5%) of the nearly 20 percent of Chrysalis 
students who had reported their first experience at 12 years of age or earlier responded that it was. 
This may place considerably different interpretation on the "age of onset" figures shown in Exhibit 
4a. It could reduce (non-abusive) initiation of sexual activity by 12 years of age to as low as 10 
percent and 6 percent for Chrysalis and Research students, respectively. However, these survey 
data alone are not sufficient to deduce this reduction with certainty. Other sexual encounters (other 
than the first) prior to the age of 1 2 may not have been abusive, suggesting that early onset 
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prevalence rates are close to those displayed in the exhibit. Tying early sexual experiences to an 
initial abusive incident is extremely important, however, and the high prevalence rate for this initiation 
is validating to the Chrysalis program staff and administration. Laws and Gabriel (1995) summarized 
research that suggested that nearly one in three adult women had been victimized by sexual abuse 
at one time in their adolescent or pre-adolescent lives. Evidently, the Chrysalis program is targeting 
this special population quite well. 

For both groups, the rate of students reporting having four or more sexual partners in their lifetime 
increased by about ten percent between the fall and spring surveys. Altogether, there were very few 
project students who reported having four or more sexual partners in the past three months; 
however, this percentage increased slightly for Chrysalis students in the spring. 



Exhibit 4.11b 

Repeated Measures ANOVA for Sexual Activity 





Lifetime 

Prevalence 


Age of Onset 


Number of 
Sexual Partners 


Number of 
Sexual Partners 
in Past 3 Months 


- - ' 

iSMairiEffe^/- 




t 'A 








Group 


.21 


.648 


.24 


.627 


1.92 


.168 


.00 


.944 


Risk Level 


4.31 


.015* 


1.55 


.215 


6.33 


.000* 


3.35 


.037* 


Time 


1.20 


.003* 


10.04 


.002* 


16.63 


.000* 


4.97 


.027* 




jf-: : ' ----- 


Group X Risk 


1.00 


.372 


1.40 


.251 


,18 


.909 


.30 


.739 


Group X Time 


.42 


.515 


3.76 


.054 


.04 


.838 


1.65 


.201 


Risk X Time 


1.07 


.346 


.64 


.527 


.59 


.625 


.52 


.593 




Group X Risk x Time 


1.57 


.212 


1.80 


.169 


1.02 


.386 


.99 


.373 



* p<.05 



The ANOVA results include statistically significant Time main effects for lifetime prevalence, age of 
onset (p<002), and number of partners in both lifetime (p<001) and the past three months (p<.027). 
The absence of Group by Time interactions (with the possible exception of "age of onset," marginally 
significant at p<.054) indicates that the significant increases are true for both groups. Risk level is 
also a highly significant main effect for lifetime prevalence (p<.015), and number of sexual partners 
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in lifetime (p<.001) and in the past three months (p<.027). This main effect evidences the well- 
established relationship that indicates students at higher risk levels have been and continue to be 
more sexually active. 

It is somewhat curious that there are such strong fall/spring differences in the two "lifetime 
prevalence" indicators. Interpreted literally, this means that these increases (a 5 percent increase 
in the percentages of students who had ever had sexual intercourse, and a 10 percent increase in 
the number who had more than four partners) occurred for both groups during this school year. As 
noted earlier, however, this may represent a greater willingness to disclose this information with 
another year's development. 

As with alcohol and marijuana use, the level of Chrysalis program participation is significantly related 
to the prevalence of sexual activity in the past three months. Significantly fewer of those young 
women who participated in more support groups during the year were sexually active in the three 
months prior to the spring survey. Again, the message to program staff is clear. 

The prevalence rates of selected consequences of sexual activity, as reflected in pregnancy and 
sexually transmitted diseases (STDs), are displayed in Exhibit 4.12a, followed by results from the 
repeated measures ANOVA in Exhibit 4.12b. 
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Exhibit 4.12a. Consequences of Sexual Activity 
Year 2 Fall and Spring Results for Chrysalis and Research Groups 




Chrysalis Research Chrysalis Research 

Pregnant One or More Told by Doctor had STD 

Times in Lifetime 



Based on students taking both Fall and Spring surveys and 
(Chrysalis) participating in at least 1/3 of the program activities 
(Chrysalis N-76; Research N-96) 



Exhibit 4.12b 

Repeated Measures ANOVA for Consequences of Sexual Activity 





Pregnant During Lifetime 


T old by Doctor Had STD 
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,/;'Signfficar^ 


Group 


3.62 


.059 


2.34 


.128 


Risk Level 


11.43 


.000* 


2.32 


.078 


Time 


1.86 


.174 


.16 


.692 
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Group X Risk 


.06 


.983 


1.34 


.263 


Group X Time 


.84 


.360 


1.09 


.298 


Risk X Time 


.70 


.554 


.49 


.692 




Group X Risk x Time 


1.69 .172 


1.14 


1 .333 1 



* p<.05 
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The percentage of project students who reported having been pregnant at least once in their lifetime 
increased at about the same rate for both groups from fall to spring, this rate was higher for 
Research students than Chrysalis students. In the spring, about three out of ten Chrysalis students 
and four out of ten Research students claimed having been pregnant at least once. These rates are 
four to five times those of their high school female peers across the nation (CDC, 1995). 

The differences between the two groups at both points in time were marginally statistically significant 
(p<054), according to results from the repeated measures ANOVA illustrated in Exhibit 4.12b. The 
Risk Level main effect is highly significant (p<.001), meaning rates of pregnancy differ by risk level 
— students with more risk factors were more likely to have been pregnant than students with fewer. 
This difference is similar for both groups, however, as there is no Group x Risk interaction. 

Almost twice as meiny Research students compared to Chrysalis students reported having been told 
by a doctor they had a sexually transmitted disease. The percentages for both groups remained 
exactly the same at both data points, and none of the difference between the groups proved to be 
statistically significant from the repeated measures ANOVA. 

Suicide Ideation and Attempts 

Mental health concerns are paramount to the Chrysalis program. As shown in Exhibit 4.13a, a high 
percentage of project students reported suicide ideation and attempts. By the end of the school 
year, more than four out of ten Chrysalis students reported that they considered suicide at some time 
during the past year. About one out of three Research students reported this to be true for them. 
These numbers are high for both groups, but there is not a statistically significant difference between 
the groups, as determined from the repeated measures analysis of variance results displayed in 
Exhibit 4.13b. 



ERIC 



Page 54 



65 



Portland Public Schools ■ RMC Research 



Exhibit 4.13a 

Prevalence of Suicidal Behavior 

Year 2 Fall and Spring Results for Chrysalis and Research Groups 
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Chrysalis Research Chrysalis Research Chrysalis Research Chrysalis Research 

Seriously Planned Attempted Attempted with Injury 

Considered 



Based on students taking both Fall and Spring surveys and 
(Chrysalis) participating in at least 1/3 of the program activities 
(Chrysalis N=76; Research N=96) 



Exhibit 4.13b 

Repeated Measures ANOVA for Suicidal Behavior in Past Year 
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.677 


.41 
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The proportion of Chrysalis students who reported having planned to commit suicide during the past 
year essentially was the same between the fall and spring. The percentage of Research students 
stating this was less in the spring (24.0 percent) than in the fall (33.3 percent). This decrease was 
not large enough to create a statistically significant difference between the groups (Exhibit 4.13b). 

The percentage of project students who claimed to have attempted suicide at some point during the 
past twelve months is essentially the same for both groups, and remained about the same 
throughout the school year. Again,- results from the repeated measures ANOVA indicate the rates 
of attempted suicide for both Chrysalis students and Research students was essentially the same. 

Chrysalis students who reported attempting suicide in the past year which resulted in an injury 
dropped from 7.9 percent in the fall to 3.9 percent in the spring. This indicator also dropped for 
Research students, but by a lesser amount: 9.6 percent to 7.4 percent. Because the actual 
prevalence rates for this indicator are so small, the differences between the groups are not 
statisticailly significant (see Exhibit 4.13b), but because of the seriousness of this indicator, program 
staff justifiably feel the reduction in suicide with injury exhibited by Chrysalis students is important. 

Among all ANOVA effects, only the Risk Level main effect was statistically significant and indicated 
that students at higher levels of risk were more likely to engage in suicide ideation and attempts. 
Interestingly, this effect grows progressively less significant as the questions go from suicide ideation 
("seriously considered," p<.023) to serious attempts ("attempted with injury," p<.9l 1). Apparently, 
serious suicidal behavior is a more complex phenomenon than current risk factor theory can predict. 

Grade level differences show a similar pattern. While older students are rnore likely to report 
considering suicide (p<.037), differences in planning and attempting are marginal (p<.079 and 
p«c.086, respectively) and in attempts with injury non-existent (p<.948). Again, describing this four to 
ten percent of students who have actually attempted suicide and suffered an injury in their attempt 
is not a simple task. Further analysis of the wealth of data collected in this study, both quantitative 
and qualitative, should be done to better understand this serious phenomenon. 

Violent Behavior 

Exhibit 4.14a displays the prevalence of physical fighting and weapon carrying by Chrysalis and 
Research girls in Year Two. Physical fighting among project students declined during the school 
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year, especially among Chrysalis students. About 41 percent of both Chrysalis and Research 
students in the fall said they had been in a physical fight during the past year. In the spring survey, 
this number dropped to 29.3 percent for Chrysalis students and 33.3 percent for Research students. 
Since the rate dropped fairly consistently between the groups, there is a statistically significant Time 
main effect, but not a Group x Time interaction on this indicator (see Exhibit 4.14b). It is also 
noteworthy that Risk Level is not a significant effect here, indicating that students of varying levels 
of risk are equally likely to have been in a physical fight. 
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Exhibit 4.14a 

Prevalence of Fighting and Weapon Carrying 

Year 2 Fall and Spring Results for Chrysalis and Research Groups 
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Past Year in Past 30 Days at School in Past 30 Days 




Based on students taking both Fall and Spring surveys and 
(Chrysalis) participating in at least 1/3 of the program activities 
(Chrysalis Ns76; Research Ns96) 



Chrysalis students reported no change in whether or not they carried a weapon to school in the past 
30 days between fall and spring. The percentage was 9.5 in the fall and in the spring for Chrysalis 
students. This percentage slightly decreased for Research students (from 1 0.4% in the fall to 7.3% 
in the spring), but this difference was not significant (see Exhibit 4.14b). 



The pattern of decreasing violent behavior shown in Exhibit 4.14a continues with the third indicator, 
"threatened with a weapon at school during the past thirty days." The percentage of Chrysalis 
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students who reported being in such a situation decreased by eight percent between the fall and 
spring data points, and the number of Research students who reported “yes” to this question also 
was reduced by five percent. Again, the pattern of these indicators is positive (violent behavior and 
situations is decreasing), but differences are not yet large enough to be statistically significant. 



Exhibit 4.14b 

Repeated Measures ANOVA for Prevalence of Fighting and Weapon Carrying 
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Attitudes, Family and Peer Health Risk Behaviors 

In addition to self reports of their own behaviors, students were asked a series of questions about 
the extent to which their friends engaged in these behaviors, their perceptions of their parents' 
attitudes toward these behaviors, their own attitudes toward these behaviors, and their abilities to 
not engage in these behaviors if they did not want to (i.e., resisting pressure to do so). 

These data, in the form of additive scales of several items on the Risk and Resiliency survey, were 
analyzed in much the same manner as the behaviors themselves reported in the preceding pages 
of this chapter. In these analyses, two major findings emerged. First, there were no significant 
differences relating to Group or Time, but there were consistent differences relating to Risk Level. 
That is. Chrysalis and Research groups did not differ, nor did these attitudes or perceptions change 
over time. But students at higher levels of risk consistently reported having more friends that 
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engaged in these behaviors, perceived their parents' attitudes to be more permissive of these 
behaviors, and exhibited more permissive attitudes toward these behaviors themseives. 

The second noteworthy finding had to do with students' seif-reported abiiities to refuse to engage 
in these behaviors if they did not wish to. Whiie there were no significant differences between 
Chrysaiis and Research giris in generai, the Chrysaiis students who received higher dosage ieveis 
of the overaii program reported significantiy stronger "refusai skiiis" than those who participated at 
lower levels. 

Protective Factors 

The Risk and Resiliency Index includes 13 subscales assessing protective factors in four areas: 
Personal Competence, Social Competence, Social Bonding, and Caring/Support. Repeated 
Measures analyses of variance were conducted on these protective factors as described for the 
previous health risk behaviors section. The repeated measures ANOVA design tests for statistically 
significant differences in three main areas: over time (between the fall and spring assessments), 
between Chrysalis and Research students, and among grade and risk levels. The design also tests 
for two-way and three-way interactions among these different areas. For example, if the two-way 
interaction between Group and Time is significant, then Chrysalis students have changed differently 
over time as compared to Research students for the protective factor being tested. 

Results of the ANOVAs are presented below. "F" statistics from the ANOVAs are provided to give 
some indication of the magnitude of group differences in relation to within group variability. The 
determination of whether these differences are large enough to be considered statistically significant 
and warrant further interpretation is given by the "significance level" in the exhibits. These values 
indicate the probability that the observed group differences are within the realm of chance variation, 
rather than representing true group differences. When these significance levels are as low as .05 
or .01, researchers typically conclude that the group differences indeed are authentic and merit 
interpretation. 

Repeated Measures ANOVA results for the Group x Grade Level x Time design are presented first 
in Exhibits 4.15 to 4.18 in order to feature any developmental differences (found in other studies 
using these scales) across the grades in these psychosocial attributes. The results from the Group 
X Risk Level x Time design follow the Grade Level presentation in Exhibits 4.19 to 4.22. 
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In order to best determine the impact of the program, Chrysalis students included in the following 
analysis are only those who participated in at least one-third of all program activities. As in the 
previous analyses of health risk behaviors, it was felt by program staff that this represented a 
minimally sufficient "dosage" level for any student to have realized the intended benefits of the 
program. Also, these data represent students who responded to both the fall and spring surveys. 



Exhibit 4.15 

Repeated Measures ANOVA by Grade Level 
Protective Factors; Personal Competence 
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The Personal Competence composite scale — composed of Self-Concept, Self-Control, Positive 
Outlook, and Self-Efficacy subscales — evidences significant differences across Grade Levels 
(p<009) and Time (p<.039). Both of these trends represent essentially developmental differences 
in these young women and are characteristic of both Chrysalis and Research groups. As they 
progress through their adolescent years (and even across a single school year), they tend to grow 
in their sense of personal competence. Among the subscales, Self-Efficacy also evidences a 
significant Grade Level main effect ) p<.029). 
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One Group-related effect is also significant, the Self-Concept subscale (p<.018). Across both time 
periods, Chrysalis participants report higher self-esteem than do their Research group counterparts. 
Ideally, for program purposes, this would have shown itself as a Group x Time interaction, indicating 
that the two groups started at the same level of Self-Concept, but the program participants gained 
more during the year. There is suggestive evidence that this may be occurring (Group x Time 
interaction significance level is p<.134), but the effect is not strong enough to warrant serious 
interpretation. 



Exhibit 4.16 

Repeated Measures ANOVA by Grade Level 
Protective Factors: Social Competence 
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In the Social Competence area, the overall composite scale shows no significant difference, although 
the Time main effect (p<.07), reflecting fall to spring differences for both Groups and all Grades 
approaches statistical significance. Among the subscales. Assertiveness shows the same kinds of 
developmental trends as were seen in the Personal Competence area. As these young women get 
older, they are becoming more assertive — again, a highly desirable outcome, but one not found 
differentially for Chrysalis participants in relation to Research group participants. 
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Exhibit 4.1 7 

Repeated Measures ANOVA by Grade Level 
Protective Factors; Social Bonding 
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Two aspects of positive Social Bonding are part of the Risk and Resiliency scale: bonding to School 
and bonding to Family. The Chrysalis program has had no significant effect on these bonding 
processes, but again, there is some evidence that there are developmental differences operating 
here. The significant Time main effect on Family Bonding indicates that both groups are feeling more 
bonded to their families over the course of the school year. Since the change is not so significant 
across Grade Levels (p<.153), it is less likely to indicate a developmental process through 
adolescence, but rather one due to the events and progress made in a single school year. 

In Exhibit 4.18, the Repeated Measures ANOVA for Protective Factors of Support/Guidance, none 
of the effects — developmental or Group-related — yield any significant differences. 
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Exhibit 4! 18 

Repeated Measures ANOVA by Grade Level 
Protective Factors: Support/Guidance 
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As with the health risk behaviors, a similar Repeated Measures ANOVA design was conducted 
replacing Grade Level with Risk Level. During intake, project students were asked about whether 
or not they had experienced a series of risk factors. For the analysis, students were categorized into 
one of three levels depending on their number of risk factors: low, medium, or high risk level. The 
following exhibits illustrate the results of how the protective factors are associated with Risk level and 
Time for Chrysalis and Research students. Again, only students who responded to both surveys, 
and out of the Chrysalis students, only the "active" ones (those who met the one-third participation 
criterion) are included in the analyses. 

In general, the results are similar to those reported in the Grade Level design, in that no Group- 
related effects are shown to be significant. Several of the protective factor subscales — particularly 
in the Social Competence area — are found to be significantly different across levels of risk (i.e., a 
Risk Level main effect). These are always in the theoretically established direction — higher levels 
of risk among these students are associated with lower levels of protective factors. As shown in the 
following four tables, students at high risk levels have a lower self-concept, are less assertive, less 
self-confident, less cooperative, feel less bonded to family, and feel less guidance from supportive 
adults than do their lower risk peers. Correlational analyses (not shown here) indicated that these 
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relationships were stronger among Research group students than Chrysalis students, suggesting 
that the project may be interrupting the somewhat natural, inverse relation between risk and 
protection — a highly desirable outcome. In the context of the current repeated measures ANOVAs, 
this would have evidenced itself in significant Group x Risk interactions (showing that differences 
in protective factors across Risk Level groupings afe not the same for Chrysalis and Research 
groups), but only one of these even approaches marginal levels of statistical significance (e.g.. 
Family Bonding at p<104). The discrepancy in the correlational and ANOVA results is likely due to 
the more precise measurement of risk in the former (the specific number of risk factors present) as 
opposed to a more general grouping (high, medium, low) in the ANOVA. 



Exhibit 4.19 

Repeated Measures ANOVA by Risk Level 
Protective Factors: Personal Competence 
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Exhibit 4.20 

Repeated Measures ANOVA by Risk Level 
Protective Factors: Social Competence 
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Exhibit 4.21 

Repeated Measures ANOVA by Risk Level 
Protective Factors: Social Bonding 
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Exhibit 4.22 

Repeated Measures ANOVA by Risk Level 
Protective Factors: Support/Guidance 
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In short, although there is extensive literature and philosophical value that enhancing these 
protective factors is a key step in reducing the negative health behaviors under study here, these 
attributes are very difficult to change — particularly in such a population as Chrysalis is dealing with. 
Program staff are among the most cognizant of this difficulty, asserting that it takes years to begin 
to ameliorate the effects of abuse in the lives of these young women. Research summarized in 
establishing the framework for this project (Laws & Gabriel, 1995; Benard, 1991) certainly supports 
their assertion. 

Intercorrelations 

Many of the significant interrelationships between program participation and health risk behaviors 
among Chrysalis women have been reported in the context of the previous discussion. The full set 
of correlational analysis is presented in this section. 



Health Risk Behaviors 



Responses from Chrysalis students on their health risk behaviors in the spring 1996 survey were 
correlated with the different program activities in which they participated to assess how and which 
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program activities impacted these behaviors. Both the program activities and the health risk 
behaviors were coded to increase with participation and use/involvement, so that a negative 
correlation means the more participation in the program activity, the lower the prevalence of that 
health risk behavior. For example, a negative correlation between support group and alcohol use 
in the past 30 days would ngean the more support groups ai Chrysalis student attended, the lower 
their reported use of alcohol in the past month. Exhibit 4.23a presents all correlations larger than 
.20; this level of correlation is close to that required for conventional .05 levels of statistical 
significance. Because sample size varies somewhat for each individual correlation, statistical 
significance criteria will as well. The authors determined that using a consistent threshold for 
correlational interpretation would be useful. 

Open sessions, composed of the HIV/AIDS Education and Media Literacy components, are the 
program activities that impacted Chrysalis students the most, particularly in the area of sexual 
activity. From this analysis, it is difficult to determine whether or not it was the program activities that 
encouraged students to lower their involvement in these behaviors or if students who already were 
less involved in these behaviors were the ones who attended these particular sessions. There is a 
positive significant correlation between case management hours and attempted suicide and also 
between case management sessions and carrying a weapon to school. It is likely these are due to 
case managers spending more time with students who were the most troubled. 

A similar correlational analysis was conducted to examine the association between level of risk and 
the health risk behaviors, as well as dosage level and health risk behaviors. Correlations greater 
than .20 for the health risk behaviors and risk level and dosage levels are shown in Exhibit 4.23b. 
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Exhibit 4.23a 

Intercorrelations Among Health Risk Behaviors and Program Activities 
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Note: * p<.01 , ** p<.001 , and other correlations larger than .20 are approaching conventional .05 levels of statistical significance 
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Exhibit 4.23b 

Intercorreiations Among Health Risk Behaviors and Risk Level & Dosage Level 



Heal 


th Risk Behaviors 


Risk Level 


Dosage Level 


Alcohol 

•Use 


Past 30 Days 


- 


— 


Number of Drinks 
When Drank 


- 




Cigarette 

Use 


Past 30 Days 


- 




Number Smoked 
When Smoked 


- 


-.28 


Marijuana 

Use 


Past 30 Days 


.29 


- 


Amount Used When 
Used 


.20 


- 


Sexual Activity and 
Consequences 


Ever Had Sexual 
Intercourse 


- 


-.24 


Age Onset 


- 


- 


Number Of Partners 
in Lifetime 


- 


-.36** 


Number Of Partners 
Past 3 Months 


- 


- 


Ever Been Pregnant 


- 


-.31* 


Ever Told by Doctor 
had STD 


- 


-.22 


Violence 


In Physical Fight in 
Past 30 Days 


- 


- 


Carried Weapon 


- 


- 


Been Threatened 


- 


- 


Suicide Ideation 
and Attempts 


Thought of Suicide 


- 


- 


Planned Suicide 


.22 


- 


Attempted Suicide 


- 


- 


Attempted Suicide 
with Injury 


.29 


- 



Note; * p<.01 , ** p<.001 , and other correlations larger than .20 are approaching .05 levels of significance 



Interestingly, risk level is negatively correlated with lifetime prevalence of sexual activity and ever 
having a sexually transmitted disease. Also, out of the Chrysalis students who said they smoked 
in the past month, those with a higher risk level smoked fewer cigarettes when they smoked. 
Dosage level was negatively correlated with number of sexual partners in lifetime and having ever 
been pregnant. 
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Protective Factors 



Program activities were also correlated with the protective factors to examine the relationship 
between them. As shown in Exhibit 4.24, none of the associations were statistically significant. The 
relationships between program activities and risk level and dosage level are also presented in this 
exhibit: there were no statistically significant correlations between risk or dosage level and the 
program activities. 



Exhibit 4.24 

Intercorrelations Among Protective Factors and Program Activities 
& Among Risk Level and Dosage Level 





Support 

Group 


Open 

Session 


Chalinge 

Course 


Girls 

Emprmt 


Yr. End 
Celebrtn 


HIV 


Media 


Case 

Mgmt. 

Hniirs 


Case 

Mgmt. 


Risk 

Level 


Dosage 

Level 


Social 

Bonding 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


School 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


Family 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


Personal 

Competence 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


Self-Concept 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


Self-Control 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


Positive 

Outlook 


- 


- 


- 




- 


- 


- 


- 


- 


- 


- 


Self-Efficacy 


- 


-.26 


- 


- 


- 


- 


- 


- 


- 


- 


- 


Social 

Competence 


- 


- 


- 


- 


- 


- 


- 


- 




- 


- 


Assertiveness 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


Confidence 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


Cooperation 


- 


- 


-.20 


- 


- 


- 


- 


- 


- 


- 


- 


Caring/ 

Support 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


Nutu ranee 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


Guidance 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 



Note: The correlations larger than .20 are approaching conventional .05 levels of statistical significance. 
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student Achievement 

Another aspect of the study was to investigate students' academic achievement while involved in 
the Chrysalis Project. Although the project is not designed to directly impact basic skills, it is 
important to examine the effects of program participation on student learning. 

Portland Public Schools maintains a Student Master Database which includes comprehensive 
information on each student enrolled in the district. Some of these data were particularly relevant 
to the Chrysalis program and were thus extracted for inclusion in this evaluation. Specifically, three 
indicators were used; 1) Graduation Standards Test (GST) scores in reading and mathematics^ 2) 
school attendance as total days absent for Year One and Year Two; and 3) grade point average for 
these two data points. 

The Portland Graduation Standards Tests (GST) were used to measure student achievement in 
reading and mathematics. The GST tests are administered each spring to all district high school 
students who have not met the minimum graduation standard in reading (RIT score of 212) and 
mathematics (RIT score of 222). 

Exhibit 4.25 displays the means for the student achievement variables. Chrysalis and Research 
students do not differ significantly in their school performances. Both groups maintained a “C” 
average during Year Two. Chrysalis students raised their grade point averages from Year One to 
Year Two (1.52 to 2.19) while Research students g.p.a. made only a modest gain from 1.9 to 2.0. 

School attendcince data also show a positive trend among Chrysalis participants as compared to the 
Research group. The mean number of days absent among Chrysalis young women was reduced 
from 36.1 days absent in Year One to 26.7 days in Year Two. Among Research youth attendance 
was unchanged; 27.4 days absent in 1995 and 26.3 days absent in 1996. 

Analysis of achievement scores and attendance data indicates that participation in Project Chrysalis 
positively affects student learning in the basic skills and improves school attendance. In fact, the 
significant difference noted indicated that reading and mathematics achievement improved with 
higher levels of participation in the program. 

The percent of project students who passed the District’s Graduation Standards Test (GST) in 
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Reading decreased from 1995 to 1996. Students in both the Chrysalis and Research groups were 
more academically needy in Year Two. Fewer Chrysalis students (84.1 %) had passed the Reading 
GST test in 1996 than the previous year (94.3%). The percentages of Chrysalis and Research 
students meeting the graduation standard in reading and mathematics are equivalent. Among 
Research students, 68.5% have passed the Mathematics test (slightly lower percentage the previous 
year). The percent of Chrysalis students who passed the Mathematics test was essentially 
unchanged from 66.5% in 1995 to 67.3% in 1996. The large discrepancy between the reading and 
math scores among project students, not seen among the districtwide female population, leads us 
to ask if more high risk girls have bought into the gender role stereotype of women as better 
reading/verbal achievers and lower in mathematics achievement. 



Exhibit 4.25 

Comparison of Academic Achievement, Year Two 

Results for Chrysalis and Research Groups 





Meain Grade Point Average 


Chrysalis 


1.521 


2.186 


Research 


1.903 

1 


2.004 


1 Mean Annual Days Absent | 


Chrysalis 


36.1 


26.7 


Research 


27.4 


26.3 


Percentage Passing Graduation Standards Test Reading. 


Chrysalis 


93.3% 


84.1% 


Research 


85.9% 


84.9% 


Percent Pacing Graduation Standards Test — Mathematics 


Chrysalis 


66.5% 


67.3% 


Research 


72.1% 


68.5% 



Exhibit 4.26 compares the means on academic indicators from the District Database for Chrysalis 
Program students by dosage level. These indicators provide evidence of significant differences in 
academic performance among Chrysalis students when dosage level is considered. High dosage 
students scored the highest on all academic indicators. In terms of achievement test scores, 
attendance, and grade point average, students in the high dosage group are significantly higher than 
students in both low and medium dosage levels. The medium dosage group has higher attendance 
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and math achievement than the low dosage group. These results show a trend towards significant 
improvements in academic progress Chrysalis students with high levels of program participation. 



Exhibit 4.26 

Comparative Means on Academic Indicators by Dosage Level, Year Two 
Spring 1996 Results for Program Group Only 













Low Dosage (n=37) 


224.5 


226.5 


31.3 


1.1 


Medium Dosage (n=29) 


223.5 


224.5 


26.6 


1.5 


High Dosage (n=84) 


227.0 


231.4 


23.5 


2.0 



Process Evaluation 
Challenge Course 

During Year Two, the Chrysalis evaluation team added two instruments to assess change in other 
program components — the Challenge Course and Girls Empowerment sessions. RMC Research 
Corporation developed a pre/post survey for the Challenge Course to assess locus of control. The 
Girls Empowerment pre/post survey was revised to align more directly with the training curriculum. 

The Challenge Course locus of control instrument included 10 items assessing protective factors in 
four subscale areas: Personal Efficacy, Interpersonal Control, Social Competence/Confidence, and 
Social Competence/Cooperation. The mean scores on these subscales prior to attending the 
Challenge Course were compared with mean scores after the training. The analysis assessed 
whether there were any significant differences on any of the protective factors as an outcome of the 
course. The comparison used descriptive statistics to give an indication of changes which may be 
linked to the Challenge Course. Results of the analysis are presented in Exhibit 4.27. While none 
of the change comparisons approach traditionally accepted levels of significance, the results do 
indicate minimal levels of positive change on each of the protective factor subscales. The 
Interpersonal Control subscale yielded the most change (.25), along with the Social 
Competence/Confidence subscale at .23. The instrument and an item analysis are in Appendix F. 
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Exhibit 4.27 

Group Means and Standard Deviations on Chaiienge Course Survey, Year Two 
Locus of Control with Chrysalis Program Group Only 













Subscale: Personal Efficacy 


57 


2.68 


.52 


2.81 


.51 


0.12 


Subscale: Interpersonal Control 


58 


2.91 


.24 


3.16 


.20 


0.25 


Subscale: Social Competence/Confidence 


58 


3.45 


.00 


3.67 


.00 


0.23 


Subscale: Social Competence/Confidence 


58 


3.65 


.07 


3.76 


.07 


0.09 



Girls Empowerment 

The Girls Empowerment pre and post survey included 22 items to assess the girls skills, knowledge, 
and attitudes related to personal safety and self-defense. Items also compared the youths’ level of 
confidence, empowerment, and knowledge of strategies to prevent physical/sexual abuse, date rape, 
sexual harassment, and domestic violence. A pre-post analysis determined whether there were any 
differences in knowledge, skills, and attitudes as an outcome of the course. 

Exhibit 4.28 displays the outcomes for the 1995-96 Girls Empowerment with Chrysalis students. 
Over half of the Program girls attended Girls Empowerment (n=96); this is lower than expected 
because the level of program participation or dosage was problematic during Year Two. The results 
indicate that participants gained new skills and confidence in all 22 categories assessed. About 90 
percent of the young women reported that they learned new skills and knowledge. Over 82 percent 
of the youth reported that they gained new awareness of their own strength and felt capable and 
empowered. The greatest increases in knowledge were in these areas: how to deal with date rape 

I 

and how to prevent physical/sexual abuse. Several self-defense skills were improved: body grasp 
escapes, choke hold escapes, wrist grab escapes, verbal assertiveness, and hand strikes. Over 66 
percent of the girls report they would take an advanced self-defense class. A detailed summary of 
the Girls Empowerment course is in Appendix F. 
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Exhibit 4.28 




Supplementary Research Activities 

The Validity Study of Seif-Reported Drug Use 

As mentioned earlier in Chaper 2, a special study was conducted during Year Two to validate the 
students’ self-report of certain health risk behaviors on the Risk and Resiliency index (see Validity 
Study Report in Appendix B). The study compared results of tests of hair samples gathered from 
students with their self-reported use of drugs. Of the illicit drugs included on the Risk and Resiliency 
Index, the hair sampling procedure could sensitively detect use of marijuana, inhalants, cocaine, 
amphetamines and heroin. Among the students who participated in the validity study, self-reported 
prevalence rates of all substances but marijuana were so low (less than 5% per group or one or two 
students using) that an assessment of the agreement between the hair analysis and self-report 
would be highly positively biased. That is, when marginal usage is this low, any sort of statistical 
index of agreement will indicate high agreement, due to the preponderance of negative results. 

With this in mind, the validity results reported here will focus on marijuana use. In all, 94 young 
women participated in the hair analysis process. Of these, 7 submitted hair samples that were too 
small to adequately analyze, and another 3 students did not respond to the 30-day marijuana use 
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item on the survey. This left 84 Chrysalis and Research students for whom both a valid hair analysis 
result and a response to this survey item was obtained. Results of the initial comparison of these 
data are shown in Exhibit 4.29 below. 



Exhibit 4.29 

Results of Hair Analysis and Self-Reported Marijuana Use 







Physiological Test of Marijuana Positive 3 2 


5 


Use (Hair Analysis) 

Negative 57 22 


79 


Total 60 24 


84 



Although the results of this comparison are ceist in a very simple table, the interpretation can become 
quite complex. The agreement of the two assessment procedures (self-report and hair analysis) is 
represented by two cells — those students found to be users by both assessments (n=2), and those 
for whom both found to be non-users (n=57). Combining these two, there is just over 70 percent 
agreement between physiological and self-reported assessments of marijuana use. 

Another approach is to focus on the “user” samples from each procedure. Of the 24 students who 
reported some use of marijuana in the past 30 days on the survey, only two of them also were 
detected as positive by the hair analysis. This can be represented as an approximately 8 percent 
agreement rate. Conversely, of the five students identified positively by the hair analysis, two 
indicated they had used marijuana in the past 30 days on the survey. This may be expressed as 
a 40 percent agreement. 

One of the circularities operating within a validity study is that neither one of the assessment 
procedures necessarily provides the “true” picture of whether a student has used marijuana or not. 
Hair testing is a promising technique, but it is not flawless. Self-report surveys, while carefully 
controlled in the Chrysalis evaluation, are also often questioned. Among other things, this makes 
it problematic to choose from among the agreement indices described above. That is, if one could 
be considered the “correcf procedure, it would suggest we calculate the percentage of times the 
other assessment results agree with it. 
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Even with this uncertainty, and options for expressing agreement, the overall correspondence 
between the two procedures is not encouraging. Looking beyond their agreement, and interpreting 
the marginal frequencies of marijuana use from the two procedures suggests one possible 
explanation. Of those students included in this analysis, over 28 percent (24 of 84) indicated they 
had used marijuana on the survey; while the hair analysis procedure detected only about six percent 
— less than one-fourth of the self-report total. This discrepancy indicates that, either the girls were 
over-reporting on the survey or the physiological procedure is too conservative. The first explanation 
is unlikely in that this prevalence rate is comparable to other survey results obtained from other 
samples in other locales. To explore the second, the RMC Research team consulted with 
Psychemedics to see if it was reasonable to lower the detection threshold on the hair analysis to 
determine whether an appreciable portion of the self-reported users who had tested negative on the 
initial test would test positive with a lower quantity of marijuana evidenced in their hair sample. 

Results of this secondary analysis yielded an additional nine students who tested positively at this 
lower level, and another five who “just missed” at this threshold. Adding the nine to the “positive” 
category brings the agreement rate among self-reported marijuana users to just under fifty percent. 
Adding the additional five who “just missed” brings the agreement to about two-thirds. Lending-their 
experience to this, Psychemedics staff were clearly more comfortable with including the positives 
from the lower threshold than they were with the “just missed” category. 



Applying this adjustment to the previous exhibit reconfigures the Exhibit 4.30 as follows: 



Exhibit 4.30 

Results of Hair Analysis and Self-Reported 
Marijuana Use After Lowering Threshold of Hair Analysis Test for Students 
Who Had Reported “Any Use” on the Self-Report Survey 
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While these results show improved agreement between the two assessment procedures, they are 
still not terribly impressive from a statistical point of view. Since the hair samples from the self- 
reported non-users were not retested using .the lower threshold, we cannot compute overall 
agreement indices with any certainty. At best (i.e., if all self-reported non-users still tested negative), 
the overall agreement index would be 81 percent — comparable to the 79 percent agreement 
between urinalysis and self-reported marijuana use in a criminal justice population (National Institute 
of Justice, 1991), and better than many found by Mieczkowski (1990) in his comprehensive review 
of self-report validation studies. This appears to be a high figure, but one that still indicates 
approximately one in five students have used meirijuana according to one method, but have not used 
according to the other. 

We remind ourselves that this was a highly experimental procedure, using a relatively non-intrusive 
physiological measure to validate self-reported marijuana use. The literature suggests promise for 
hair analysis, although it is not yet recognized by the National Institute on Drug Abuse (NIDA) as a 
valid technique. Other researchers (Harrison, 1995) have pointed out that it is highly preferred to 
urinalysis due not only to its relatively non-intrusive nature, but also because its window of detection 
of drug use is so much wider. Given that the secondary analysis results here could yield agreement 
rates comparable to those found in urinalysis studies, hair analysis may be a reasonable method for 
use in a school population. 

We suspect that there is still much to learn in the implementation of such a procedure, however. Our 
consultations with the professionals at Psychemedics indicated to us that there are still many key 
decisions to make in setting standards and criteria for the physiological testing, particularly when 
working with this population. It is reminiscent of the decisions the applied researcher makes in 
setting acceptable probabilities for Type I (false positive) and Type II (false negative) errors. 

HIV/AIDS Education Project 

The HIV/AIDS education supplement was added to the existing Chrysalis program in order to test 
the hypothesis that delivering an HIV/AIDS education curriculum within an existing prevention 
program would be more effective for participants than providing a HIV/AIDS curriculum that was 
outside of a prevention program. To test this hypothesis, the Chrysalis Research group was used 
as the nonprevention treated comparison group. 
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The initial implementation design for the HIV/AIDS curriculum was for both Program and Research 
participants to have the curriculum presented in four separate sessions during the school year. The 
Chrysalis Program students had prescheduled Open Sessions which made it convenient to insert 
the curriculum into those time slots. Scheduling four sessions for the Research group was more of 
a challenge, however, as they had no regular meeting times. Program staff decided that it would be 
more efficient to use an all day retreat format for the Research girls in order to resolve the scheduling 
challenge. 

The retreat format turned out to be the most effective way of delivering the HIV/AIDS curriculum. 
The HIV/AIDS curriculum could be presented all at one time, and thus the girls who attended did not 
miss any of the important components of the program. 

With this change in implementation and the positive results that change brought to the Research girls 
in the form of more efficient and effective delivery of the curriculum, assessing the differential 
effectiveness of a HIV/AIDS curriculum delivered within an existing prevention program compared 
to a nonprevention treated comparison group could no longer be done. 

The primary tool for assessing increased knowledge and preventative attitudes about HIV/AIDS 
among Chrysalis participants was a survey developed by the HIV/AIDS education specialist and the 
evaluation staff. The first seven questions on the survey deal with how the HIV/AIDS virus is 
transmitted (knowledge): the last 10 questions ask about the change in intentions of behavior to 
protect against HIV/AIDS (attitudes). 

Analysis by dosage level finds that Program girls who attended two or more HIV sessions had 
improved outcomes and gave a higher percentage of appropriate responses in five of the first 7 
items when compared to Research girls. If the survey results are compared between all Chrysalis 
students and Research students, then a higher percentage of most appropriate responses were 
found in the Program girls for only three of the seven questions. Thus, dosage appears to be an 
important factor in showing a positive impact on the girls' knowledge of HIV/AIDS and its prevention. 

The remaining ten survey items measure protective behaviors against HIV/AIDS. Again, analysis 
by dosage level shows that Program students who participated in two or more HIV sessions had a 
higher percentage of appropriate responses to four of the ten survey items than Research girls. Four 
additional questions showed percentages about equal to the Research girls, and only two questions 
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showed lower percentages. Whereas, when not looking at the dosage level percentages, research . 
girls were more likely to give the most appropriate response — doing so for seven of these items. 

The Program and Research comparisons were based on post test scores only, assessing students' 
knowledge and attitudes after they had received the HIV/AIOS training. Within the Research group, 
we had the opportunity to document the change in the outcomes from the pretraining survey. A 
comparison of pretest and posttest responses for the Research girls shows substantial increases 
in appropriate responses following exposure to the curriculum. 

When comparing Chrysalis and Research groups, survey responses showed that knowledge about 
HIV/AIDS contagion and prevention and intention to change behavior to protect against HIV infection 
was equal to and often times higher for Chrysalis girls who had attended between two to four 
curriculum sessions than Research girls who had received the entire curriculum at one time. 

Overall, both the intervention and control groups benefited from the program, but the control group 
received the benefit of more effective implementation of the curriculum. In retrospect, the curriculum 
needed to be delivered the same way to both groups to yield a valid comparison of effectiveness. 

Media Literacy Project 

The media literacy curriculum and the two-week workshop were designed to increase students’ skills, 
knowledge, and attitudes related to the content and form of media messages and their impact on 
self-image, ATOD use, and health issues. It was anticipated that the media literacy training would 
empower youth to better evaluate the impact of visual/print images on women of various cultures. 

\ 

Process and outcome evaluation activities included structured observations and interviews of 
participants, survey administration and review of instructor notes and student evaluations. All of 
these activities were conducted to determine to what extent the Media Literacy Project had 
accomplished the desired effects on the target population. 

Three of the Chrysalis Open Sessions, were devoted to teaching the skills needed to identify and 
critique the messages various media disseminate regarding women (including adolescents and 
women from various cultural and socioeconomic groups), and the impact of these messages on self- 
image, use of ATOD, and other unhealthy behaviors. 
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Participants were encouraged to keep journals recording their observations of the content and form 
of various media messages. This activity was used to spur discussions and counter the tendency 
to be unquestioning consumers of media such as television and movies, whose rapid-fire images 
prevent active interaction or questioning by viewers of what they see on the screen. 

The media literacy activities provided during the regular school year also benefited the Chrysalis 
case managers and one or two high school teachers responded with an interest in learning how to 
integrate the subject matter into their classroom curricula effectively. 

Media Surveys. Pre and post surveys were administered to participants at the media literacy open 
sessions. A total of 108 surveys were completed; there were 72 pretest surveys and 36 posttest 
surveys. Of the three media sessions offered the Program group during the school year, 60 percent 
(n=114) of the 189 students attended at least one session. About one in eight (12 percent) of the 
Program girls attended all three sessions. As the surveys were not linked to a project identification 
number, these participation differences could not be related to any differences in survey responses. 

The first 13 survey questions asked participants to rate their knowledge pertaining to such things as 
television advertising and public service announcements and to rate their ability to recognize 
distortion of truth, manipulation, and biases found in the media. Statistically significant increases 
in the percentage of "very good" responses occurred when participants were asked to rate a) their 
skills as a critical viewer, b) their knowledge of how to produce a video message, and c) their ability 
to communicate prevention messages to the public. The increase in "very good" responses was also 
nearly significant for the ability to recognize media marketing to ethnic groups between pre and 
posttest. In addition, the number of participants who responded "don't know" to each item lowered 
dramatically from pre to post, indicating that familiarity with the issues had been greatly enhanced. 

The remaining ten statements measured attitudes and opinions about community exposure to drug 
and alcohol abuse information through the media and the need for this type of exposure in the 
Portland area. The posttest showed a sharp increase in the percentage of girls who agreed with 
statements indicating that Portlanders have access to friendly and humane public services and 
support media campaigns aimed at preventing drug and alcohol abuse and showed a moderate 
increase in support for additional funding for alcohol and drug prevention programs in schools. 
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Media Production Workshop. Ten Chrysalis girls were recruited to apply what they had learned 
about media literacy during the school year to designing and producing a 30-second animated public 
service announcement (PSA) and a three-minute videotape. The diverse group of students attended 
a two-week media literacy workshop in June 1996. 

The project was a collaborative work of all the students. The theme and story line of the PSA, Fight 
Girt Poisoning, was created and developed through consensus by the group. Each person had a 
role to play in the production and was responsible for getting their work done in the time frame 
allotted. Some of the girls drew the dozens of pictures needed for each frame of the PSA. Other 
girls were responsible for developing the script and the audio portions of the video. 

The three-minute video, Girt Power, required each girl to develop a small mini-story about an ATOD 
issue that was important to her, draw the pictures, and provide the script and audio for their piece. 
Each day was full of acquiring new knowledge and skills. The participants learned what animation 
was, what it entailed, and how to apply that knowledge to the development of their own animation 
work. They were introduced to a variety of film equipment and were shown how to use it. 

An important aspect of the workshop was the commitment that each participant was willing to show 
during the two-week course. All the young women were required to attend the workshop every day 
. and to arrive on time. This kind of commitment is important as the project needed everyone's input 
and work. Otherwise, the project would not have succeeded as intended. This commitment of their 
time was also significant because many of the girls had attendance problems in the past school year. 

Video Premiere. Project Chrysalis held a special showing of the PSA and short video the girls 
developed in the media literacy animation workshop for participants, friends and family and staff. 
It was held at the Northwest Film Center's auditorium, next to the Portland Art Museum, in downtown 
Portland. The PSA, Fight Girt Poisoning, and the film. Girt Power, were shown on the big screen, 
which awed and impressed not only the workshop participants but other members of the audience 
as well. The pride that the girls and the rest of the audience felt about the workshop results was 
palpable. Each girl was given her own copy of the PSA and film as well as a special award for her 
individual contribution to the workshop. 
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Case Study Findings 

The Chrysalis case studies contribute to the empirical description of the student population and the 
implementation of the project. As noted earlier, four case studies were conducted during Year Two 
of the program; two with Chrysalis Program students and two with Research Group participants. 
Cases were selected to extend the understanding of specific variables related to the target 
population and program implementation. For example, the first case studies investigated the 
relationship between childhood abuse and its impact on teen parenting, pregnancy, and alcohol and 
other drug use. 

The case studies augment the outcome evaluation by giving a voice to the student risk and resiliency 
factors. In Year Two, the case study added two new students, while continuing to follow two 
students from the pilot cohort. The study also expanded the type of case study participants to 
include additional variables. 

The following sections provide excerpts from two case studies. These case summaries describe the 
experiences of one Chrysalis Program Group participant, Kate, and a Research Group student, 
Dianne, during their time with the project. 



Case 1 — Kate 

Kate is a 17-year-old biracial young woman at Salmon High School. As a sophomore, she learned 
about the Chrysalis program for female victims of physical, sexual, or emotional abuse and referred 
herself to the school counselor. During the screening interview, Kate spoke of four characteristics 
associated with high risk: she was a victim of sexual abuse, a child of substance abusers, a lesbian, 
and economically disadvantaged. As a junior, Kate continued her active participation in the 
Chrysalis program. 

Kate is careful and thoughtful in her responses to the questions that are asked of her, taking her time 
to be sure that she expresses exactly what she means. It is not only important for Kate to answer 
questions purposefully but to ask questions of herself in order to find the truth. As she says, 

/ ask myself what am I doing here and why. What do I want to do. That sort of stuff. 
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A self-described athlete, she actively takes part on her high school's soccer, tennis and basketball 
teams. In tennis, she received the most valuable player award at the high school's year end awards 
ceremony. Kate also does well academically, maintaining a high B average, but her passion is her 
music. A career in music is where she will focus her energy when she graduates from high school. 
In school, she has sung in the choir and has performed in several plays. She writes and sings her 
own songs and plays the guitar. After graduation, Kate plans to attend college, hoping to finance 
her studies with an athletic scholarship. 

Of African-American and European-American heritage, Kate lives with her father. Her parents 
divorced when Kate was three, but her mother and stepbrother moved back into the home when she 
was eight years old and stayed for five years. Kate said that this period of time was very difficult and 
stressful for both her and her father. It was during the first year of her mother's move back into the 
house that the sexual abuse occurred. Her stepbrother abused her when she was eight years old 
and he was twelve. The abuse began in the summer before Kate started third grade and lasted for 
several months until she told someone at school who had given a presentation to her class about 
what a "good touch, bad touch" meant. It took awhile to remove her stepbrother from the home, but 
he was eventually moved. 

After the abuse was recognized, Kate attended a support group for about six months. Her 
recollection of attending the support group is not clear. All she remembers is not fitting in well with 
the group. She did not have individual counseling and felt that she did not need it. Kate doesn't 
dwell on her sexual abuse experience. To her, the abusive incident isn't as important as "the 
emotional journey I've been through." She is willing to talk about it, but feels that the story of her 
abuse doesn't "tell anybody about me as a person." Working on other issues in her life are more 
important to her right now. 

Kate is from a family with an extensive history of substance abuse on both the paternal and maternal 
side. Her mother's family, including several of her uncles, have suffered serious consequences from 
their substance abuse. It is Kate's greatest worry that she won't be able to resist using alcohol or 
other drugs at some time in her life. She worries that: 

/ might give up and become abusive of substances and people. I might just stray far 

from what I hold true now. I might become somebody I can't agree with. 
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Kate attends, along with her father, various support groups including AA, Alanon, and Alateen. She 
and her father talk openly about the influence drug abuse has had on their lives. 

Kate has a close and supportive relationship with her father and considers him to be her family. Her 
father supports them financially by repairing musical instruments. Years ago he was a musician and 
played in a band along with Kate's mother. Her mother is very much on the periphery of her life and 
is not relied on for help or approval. Kate describes her mother, who was also a victim of abuse, as 
a "flake" and a person who doesn't take responsibility for her life. 

Kate is a lesbian and feels comfortable with that reality although she has not discussed this with her 
mother and doesn't want to at this time. She does not actively declare that she is a lesbian, but will 
tell people if they ask. Kate said she has known for a long time that she was gay, but finally 
understood what it meant for her in eighth grade and that it was okay. She and her father have a 
very open relationship and discuss all aspects of her life including her sexual orientation. 

Her experience as a lesbian sets her apart from her peers. She tends not to have close friends and 
relies often on her father to be her social ally. As she says, 

/ don't talk to my friends much. It's not like a choice. They have jobs, have boyfriends 
and things. I talk to my dad a lot, and teachers. 

Kate says that there definitely is a gay/lesbian culture-a way of saying and doing things that is quite 
different from the heterosexual world she lives in. She plans to become involved in the gay 
community when she graduates from high school. 

Kate has regularly attended the Chrysalis support group meetings, participated in the Girls 
Empowerment and Challenge Course and in the program's end-of-the-year celebrations. She 
particularly liked the Challenge Course because it was physically demanding, and she enjoyed the 
opportunity to get to know her fellow group participants. 



Throughout her time in Chrysalis, Kate has not liked the support group component of the program. 
She was impatient with the group leader, who she felt was not direct in her approach in dealing with 
the topics being discussed in the group, taking too long to get to the point and thus giving the 
participants little time to take part in the group in a meaningful way. She expressed ambivalent 
feelings about coming to group. 
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/ still don't like the groups, and oddly enough, I look forward to them in a weird way even 
though they interrupt my favorite class of the day. I think it's just because I have one 
person who understands me a little bit, and we have silent conversations across the 
table or when we're sitting next to each other. Basically, it's a chance for us to make fun 
of the other people who don't understand us. I feel guilty. I feel like since I'm close to 
this person, I don't want her to be stuck with the other people, so I come for that. 



Kate attends several different support groups outside of class and feels that these meetings give her 
what she needs. At these meetings no "cross-talk" is allowed, in other words, no one interrupts, 
makes comments or gives advice to the person who is talking. She says that just the opposite 
happens at Chrysalis support groups. Others interrupt what she is saying with the end result that 
Kate doesn't have the opportunity to share what she wants to share. She says 

/ shared my story about my abuse and I said this is what I want: I want you to not 
interrupt me and I want to be able to say my thing. . .And what this girl did is she wrote 
down questions and she'd ask me a simple question, like yes or no. I'd say no and 
she'd say, 'Yeah, that's interesting because with my story, blah, blah, blah.' This is my 
time. Don't steal my fire. It pisses me off that they do that, and they do it all the time. 

So the meeting turns into their thing. 

The support group component has been a disappointing experience for Kate. There were other 
program components she found useful and interesting such as the HIV/AIDS Open Sessions . She 
especially enjoyed her participation in the Media Literacy film workshop during summer 1 996. 

In summary, Kate has many inner strengths and external supports that have assisted her in meeting 
the challenges of a background of sexual abuse and family substance abuse. A close and 
supportive relationship with her father, a comfortable acceptance of her sexual identity: a recognition 
of her own self-worth; the willingness to put her abuse behind her; an active involvement in sports; 
a passion for music; plans for continuing her education; and participation in the Chrysalis program 
have all contributed to a positive outlook for Kate's life. 

Case 2 — Dianne 

Dianne is a 17-year-old Caucasian young woman, a sophomore at Adams High School. Her school 
alcohol and drug counselor referred her to the Chrysalis Project two years ago based on her history 
of sexual and emotional abuse. At her intake interview, Dianne spoke of many experiences that put 
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her at high risk: she was a child of substance abusers, a recovering drug abuser herself, a school 
dropout lagging at least one year behind her peers in academic credits, she was experiencing severe 
mental health problems, a victim of chronic pain, and at risk of suicide. During her sophomore year, 
she was assigned to the Chrysalis Research (control) group. She was sad that she was not selected 
for the intervention, but happy to participate in the Research activities. As she met several variables 
identified for the case study, her case manager recommended Dianne as a good candidate for a 
case study and a girl who might also benefit from having another caring adult in her life. 



The young 17-year-old woman came toward me looking like a typical teen: an average-size girl with 
a face like a flower, dark eyes, and a turned-up nose covered by a sprinkle of freckles. Her 
shoulder-length dark hair vt/as pulled back casually behind her ears. She started talking animatedly 
about her friends, her favorite subject at school (band), her pets, and how much she loves to ride 
horses. This was my first meeting with Dianne 18 months ago. 



Dianne thinks carefully before she responds to questions asked of her. Yet, the more she talked, 
the less she resembled the young woman I’d read about who had lived through trauma that most of 
us never experience. Dianne is the youngest of four children; her step-siblings are 13 to 22 years 
older than her. She was raised by her mom from birth to age five, as her natural father abandoned 
the family before Dianne was born. Poverty and an unstable family environment led to early 
emotional and sexual abuse by her first step-father and a male babysitter. The experience has made 
her feel isolated from others, with only her mom as her protector and friend. She now says: 

I’d like to go back and change everything at age two. No more abuse. I wish my mom 
had divorced my old step-dad then, so we wouldn’t have to learn new patterns. 

One of her earliest positive memories is at age five when she met, and then introduced her mom, 
to the most important man in her life, “Poppy,” her step-father who died this past year: 

He lived upstairs from us. After school, I’d go up to visit and he’d be baking cakes, 
muffins, or cookies. One day I Introduced my mom to him and they got acquainted. All 
these years he has taken care of mom and me. Later, we moved In with him and he 
became my step-dad. Now we all laugh that I met my ‘Poppy’ before my mom did. 

When Dianne was 13 she began to skip school and interact with peers, who were alcohol and drug- 
involved and delinquent. Over the next two years, she drank and smoked marijuana, ran away from 
home, was pregnant, and dropped out of school. At age 15, she went to Kaiser for drug assessment 
and treatment, but soon relapsed. Her mom put her in an inpatient treatment program. Since 
completing treatment, Dianne has attended a neighborhood AA meeting every week, as she says: 
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My recovery is the most important thing to me. It's what I like best about myself. I like 
that I’m staying clean and sober and can enjoy music too! I’m so proud that I have kept 
my recovery going for almost two years [now three years]. But it’s hard to maintain. I’m 
working a lot at it still. Some days it’s okay; other times there’s a lot of stress on me and 
it’s hard not to just take a drink and escape. I wouldn’t change anything about my 
recovery. I learned a lot from it. 

During her adolescence, Dianne has focused more closely on her alcohol and drug recovery than 
on her recovery from abuse. She does not dwell on her past abuse experiences. She feels that 
working on other issues in her life [mourning her step-father who died last winter, finishing high 
school, staying dean and sober, etc.] are more important to her right now. She briefly saw a 
community-based counselor for group therapy, but dropped out when as she says: 

It [group therapy sessions] got too expensive and the issues were just too tough to 
handle right then. I might not be clean and sober today, if I had stayed in that group. 

I chose to stay with my AA recovery. When I’m older, when I’m ready, when the time 
is right. I’ll work on my other issues. 

Music is Dianne’s motivator for coming to school. She actively takes part in her high school’s band 
and, orchestra. She is a special education student who doesn’t enjoy reading or general education 
courses, but usually maintains a C average. Computers and keyboarding interest her and she talks 
animatedly of pursuing a career working with computers. She has no goals beyond graduating from 
high school, except for a notion about taking a computer course at a vocational school. 

Dianne has seen Chrysalis from both sides — from her vantage point in the Research group and 
from the perspective of her best girl friend, Allison, a Chrysalis Program participant. Allison was in 
Chrysalis for two years and graduated from high school last June due in a large part to the support 
and guidance of her Chrysalis case manager. Dianne did not graduate from high school with her 
friend, but you could see her tears as she played in the orchestra as the graduates marched across 
the stage to receive their diplomas. This fall Dianne is back in school trying to achieve her diploma. 

Her involvement with Chrysalis has both supported and disappointed Dianne. In September 1996, 
she asked her case manager and this researcher if she could be in the Chrysalis Program group this 
year. She felt that she had contributed to the case study research and she wanted to be part of the 
Chrysalis support group experience. For all involved, saying no was hard. As an alternative, she 
continues to take part in an alcohol and drug recovery support group facilitated by the case manager 
at her high school. 
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The case study will continue to follow Dianne this year to determine what resources are available 
to her and what outcomes this young woman will create for herself from her experiences. We are 
especially interested in learning what features of her life help to strengthen her own resolve and 
resilience. Uncovering these characteristics will help the learning community identify what works in 
keeping the next generation of children safe. 

Comparative Health Risk Behaviors and Protective Factors of Case Study Youth 

The preceding excerpts from two of the Chrysalis Year Two case studies give a voice to participants 
in the Program and Research Group. During Year Two, four students participated in the case study 
research. Exhibit 4.31 compares the demographic characteristics, health risk behaviors, and 
protective factors evidenced among the four case study young women. Over the course of the next 
year, we will further investigate the relationship of these variables, specifically the number and type 
of risk and resiliency factors, on positive and negative outcomes related to the program. The 
researchers anticipate that the Year Three case study analysis will provide case-ordered descriptive 
categories to link the Program Group students by level of risk, dosage level, and the level of impact 
of the program on the participants. 



Exhibit 4.31 

Risk and Protective Factors for Case Study Students, Year 2 











.DOMAIN: INDIVIDUAL ~ 

Chrysalis Group 


Program 


Research 


Program 


Research 


Age 


16 


18 


17 


16 


Grade Level 


10 


10 


11 


1 0/Dropout 


Ethnic Group 


White 


White 


BlackAVhite 


White 


Chrysalis Dosage Level, Year 2 


High 


NA 


High 


NA 


Uses Alcohol or Other Drugs 




In recovery 




• 


Uses Tobacco 




• 




• 


Violent/Delinquent Behavior 


• 








Juvenile Justice Involvement 


• 








Witness to Violence 


• 








Victim of Domestic Violence 


• 








Knows Someone Murdered 


• 








Seriously Considered Suicide 


• 


• 


• 




Severe Mental Health Problems 


• 


• 




• 


Physically Disabled/Chronic Pain 




• 






Teen Parent 








• 
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P 


i^P' 


H 








Runaway/Foster Home 


• 








Sexual Minority Youth 






• 




Sexually Active 


• 


• 




• 


Victim of Physical Abuse 


• 








Victim of Sexual Abuse 


• 


• 


• 


• . 


Victim of Emotional Abuse 




• 


• 




Has a Hobby (music, sports, crafts) 


• 


• 


• 


• 


Demonstrates Self-Reliance Skills 




• 


• 




Demonstates High Self-Esteem 






• 




Evidence of Support/Guidance 




• 


• 




DOMAIN: FAMILY 


Family History of AOD Use (COSA) 


• 


. • 


• 


• 


Low Socioeconomic Level 




• 


• 


• 


Parent with Emotional Problems 


• 






• 


Parent History in Justice System 


• 








Parent Incarcerated 


• 








Family Norms Favorable to Drug Use 


• 


• 






Lives with Immediate Family 




• 


• 


• 


Low Involvement with Extended Family 


• 


• 






Lives in Foster Care 


• 








Isolated from one or more Family Member 






• 


• 


High Expectations from Parent/Family 


• 




• 




DOMAIN: SCHOOL 


Grade Point Average, Years 1 & 2 


2.0 / 2.3 


1.7/ 3.5 


3.4 / 3.8 


3.4/ 3.5 


Total Days Absent, Years 1 & 2 


19/1.0 


47/59 


36/13 


12/22 


Chronic School Failure 




• 






School Dropout 


• 


• 




• 


Excessive Absenteeism 




• 


• 




Learning Disabled/Special Education 


• 


X. 






Plans to Attend College 


• 




• 




Involved in Clubs, Teams, Band, Sports 


• 


- • 


• 




DOMAIN: PEERS ' ■ 


Alienation and Rebelliousness 


• 






• 


Early Initiation of Problem Behavior 


• 


• 




• 


Peers Favorable to Antisocial Behavior 




• 




• 


Peer Attitudes Favorable to Drug Use 


• 






• 


Friends Use Drugs 


• 






• 


Rewards for Conventional Involvement 


• 


• 


• 
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Chapter 5. Cost-Benefit Data and Analysis 

Government funding decisions have increasingly been based on cost and effectiveness measures. 
The cost to society of alcohol and other drug use have been well-documented among the prevention 
community. But, the many positive results of effective prevention and the cost savings of these 
investments have not been convincingly communicated to the general public. Evaluation can help 
build the case for prevention by identifying and developing new ways to demonstrate cost-benefits 
and cost-effectiveness in programs which contribute to the decline in alcohol, tobacco, and other 
drug (ATOD) problems. 

In 1990, it was estimated that alcohol and other drug-related crime cost this nation over $57 billion 
in criminal justice expenditures and lost productivity. Health care brought this total to $165.5 billion 
(Johnson, 1993). That is more than $800 per year for every man, woman, and child in the country. 

Each year, over 100,000 people die from the consequences of alcohol abuse. Another 400,000 die 
from cigarette smoking; and 50,000 die from the long-term effects of passive smoking (Public Health 
Service, 1991). In addition to the number of deaths directly attributable to alcohol, tobacco, and 
other drug (ATOD) use, the link between these behaviors and traffic fatalities, violence, HIV/AIDS, 
rape, teen pregnancy and child abuse represent even greater costs to society. People who die from 
alcohol-related causes lose, on average, 26 years of their normal life span; from smoking-related 
causes, about 20 years; and from drug-related causes over 37 years (Johnson, 1993). 

At this stage of the Chrysalis program, the specific consequences and costs of health risk behaviors 
are beginning to be specified by the project team. Data presented earlier in the report provide some 
indication as to prevalence rates of these risk behaviors in the target population that ultimately lead 
to these consequences for these individuals and costs to society. 

During Year Two, the project delivered 12 months of full services. Since this first Chrysalis cohort 
began services in September 1995 and will complete the program in June 1997, we will not be able 
to speak to the degree to which the program is effective until fall 1997. Obviously, benefits cannot 
be estimated without information on program efficacy. At this interim stage of the project, when 
program efficacy has not yet been fully demonstrated, we ask why do an extensive cost analysis 
without first establishing the efficacy of program outcomes. The project staff will study these issues 
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in greater detail and provide a thorough cost analysis linked to outcomes as data becomes available. 
Exhibit 5.1 presents the cost side of the cost-benefit analysis for Year Two in reasonable detail. 



Exhibit 5.1 

Chrysalis Cost Benefit Analysis, Year Two 
July 1 , 1 995 to June 30, 1 996 





(MW 


‘V'V v'.-' „„ i^Hun. 


Amount Budgeted 
(non-restricted) 


463,613.00 


Includes CSAP HIV & Media Supplementary 
Grants; does not include carryover funds. 


Amount Expended 


386,891 .44 




In-Kind Contribution 


59,533.00 


In-Kind contributions: project director’s 

salary/fringe (.2 FTE); high school principals salary/ 
fringe (12 @.05 FTE, total .6 FTE); clerical support/ 
budget management (.2 FTE); office space, 
furniture, equipment; evaluation office space, 
computer workstation; evaluation optical scanner 
with survey design and analysis software. 


Total Year Two Costs 


446,424.44 




Adjustments ■ - : 1 


50% of Evaluation Expenditures 


38,883.50 


Half of the evaluation expenditures were excluded 
in this adjustment of project costs to reflect the 
reduced level of program evaluation more typical 
for special projects conducted by this school 
district. 


CSAP Burden 


2,733.92 


CSAP requires more reports, meetings, site visits, 
and adjustments in project products than is 
generally required for projects funded by the district 
or other agencies. This is an estimate of the 
additional burden to meet CSAP’s requirements (1 0 
days of project coordinator’s time). 


Total Adjusted Cost 


404,807.02 






Number of Program Participants 
Number of Research Participants 


189 

181 


The Research (control group) young women 
received some intervention. In completing the 
intake process, they had an extended conversation 
with Chrysalis staff, and received information and 
encouragement to access additional services from 
other school or community agencies. They also 
received an average of 47.5 case management 
contacts over 21 .8 hours during Year Two, along 
with a 1-day HIV retreat. 


Adjusted Cost per Total Participants 


1,094.07 




Adjusted Cost per Program Participant 


2,141.84 
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As described in this report, the program is designed to prevent a number of adverse outcomes for 
program participants. The evaluation is designed to measure the degree to which the program is 
successful in promoting resilience and preventing negative outcomes. As discussed above, there 
are clear costs to the individual and/or to society associated with several outcome measures which 
will be a part of the evaluation, for example: 



■ School dropout is associated with decreased individual lifetime earnings; 

■ Pregnancy and teen parenting are associated with significant health and social costs; 

■ Health and mental health problems increase treatment costs and decrease productivity; 

■ Tobacco use is associated with increased health care costs and mortality; and 

■ Contact with police, property damage, and fighting associated with drug and alcohol use 
has obvious financial and social costs. 

During the next two years we will further review the research literature related to these issues in 
order to develop a cost-benefit model appropriate to this program. 
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Chapter 6. Conclusions and Lessons Learned 

Project Chrysalis enjoyed much success at its 12 school locations during the second year of 
operation. School administrators and staff have been very supportive of this innovative program for 
high risk female adolescents recovering from child abuse. Chrysalis \was fully implemented in each 
high school in the district during Year T\wo, due in part, to a successful pilot program at eight schools 
in Year One. 

Clearly, the Chrysalis Project is dealing \with a special group of young \women. All have experienced 
physical, sexual, or emotional abuse at some time in their lives. Most of the young \women have 
experienced family substance abuse issues, chronic school failure, economic disadvantages, violent 
situations, and severe emotional problems. These conditions have certainly effected the health risk 
behaviors of these young \women. Briefly, the data presented earlier indicate that the Chrysalis 
students (either Program or Research group) are: 

\ 

■ Sixty percent more likely to be sexually actiye than high school \women nationally: 

■ Three to five times more likely to have had their first sexual intercourse at age 1 2 
or younger, and to have had four or more sexual partners in their lifetime; 

■ Five times more likely to have been pregnant or told they had contracted a 
sexually transmitted disease; 

■ Tvwice as likely to be current smokers, smoking tvi/o or more cigarettes a day; 

■ Fifty percent more likely to have attempted suicide, and tvwice as likely to have 
attempted it so seriously that an injury occurred; and 

■ Fifty percent more likely to have carried a \weapon, or be threatened by one, in 
the past 30 days. 

These results indicate just ho\w dramatically attitude and experience can stimulate positive or 
negative behavior in young \women. These, and related findings, have taught the researchers much 
about program management, fidelity of implementation, and evaluation perspectives in the second 
year of the demonstration project. It is important to note, hovwever, that all of the results at this stage 
of the project are merely associations, that is, they sho\w the presence or lack of related behaviors, 
and not the cause of the behavior. In this section, the authors \will discuss some of the major 
findings, lessons learned, and next steps for future of the project. 
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Project Management 

Lower Enrollment than Anticipated In Year 2 

As discussed in Chapter 2 of this report, the Year Two implementation of Project Chrysalis included 
serving three-quarters of the target number of students expected in a full project implementation. 
A specific consequence of this lower enrollment issue included the addition of program awareness 
presentations to introduce eighth grade girls to the Chrysalis program. Waverly therapists conducted 
several sessions in spring 1996 to broaden the program's exposure with more young women. To 
date, these sessions have brought a significant number of preliminary intakes to the program. 

Project Not Fully Staffed for Entire Budget Period 

In Year Two of the project, personnel changes were an issue. The project coordinator took a 
medical leave for five months, the project secretary took another job and several months lapsed 
before her position was filled, and an internal evaluator took a condensed work week for a three- 
month period. This was problematic because with four new school-based case managers in Year 
Two, staff continuity woul^likely have improved the fidelity of the Year Two implementation. Yet, 
throughout this time, the project director and other staff increased the level of collaboration to ensure 
that the project moved steadily forward. Various tasks were divided up and the majority of project 
tasks were accomplished. A fully staffed project was online at the start of Year Three. 

Documenting Program Participation 

As CSAP staff and associates are well aware, the level of program participation or dosage is a 
critical issue in measuring program outcomes. During Year Two, the project management team, 
evaluators, and case management staff worked through many issues related to identifying levels of 
dosage and describing program participation. These issues included revising the Chrysalis 
Activity/Attendance Log to match the curriculum objectives, linking case management activities to 
the national cross-site evaluation, and establishing process evaluation feedback to case managers 
to help strengthen the students' academic achievement. The initiation of the Chrysalis Database in 
the District evaluation office streamlined the tracking and reporting system for the project. As new 
case managers join the project in Year Three, the staff recognize the need for an Implementation 
Manual for Project Chrysalis. This product will be completed during the third year of the project. 
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Project Evaluation 

Enhancing Data Quality: the Risk and Resilience Index (pre and post) 

From a methodological perspective, the benefits of random assignment and pre and posttesting 
enable the researcher to confidently attribute any post-program differences between intervention and 
control group participants to the program itself. In Year Two, although a conceptually sound pre and 
post data collection strategy was employed, in actual practice, some weaknesses were revealed that 
limited the data analysis. For example, of the 189 Program participants in Year Two, approximately 
70 percent completed either a fall pretest or a spring posttest Risk and Resiliency Index. Of this 
group, there were only 76 students who took both the fall and spring surveys and met the program's 
minimum level of services by participating in at least one-third of the program activities. Some 
reasons for this included high student absenteeism, an extended intake period, a limited time for on- 
site data collection, and survey administration procedures that informed students of the voluntary 
nature of the survey and permitted them to skip any questions that might be sensitive to them. 
Given this option, approximately one-third of the students did not respond to all survey items. 

To counteract this bias and improve the consistency and stability of the pre-post data collection 
efforts over time, the project revised the data collection plan for Year Three. At this writing, the 
program is in the early stages of its third year. The Risk and Resiliency pretest is now administered 
to each student individually as part of the intake process which includes: 1) student intake interview, 
2) informed consent forms are signed by student and parent, 3) completion of the pre-program Risk 
and Resiliency Index and the CSAP National Youth Survey for the national cross-site evaluation. 
These materials are then delivered to the internal evaluator for random assignment to the Program 
or Research group. While the new data collection process necessarily lengthens the intake, it will 
ensure that all students have a baseline survey before their random assignment to the intervention 
or control group. Quality data is the backbone of this program and provides all involved with the best 
chance of demonstrating the effectiveness of the intervention program. Program and evaluation 
staff will continue working together to realize the benefits of this substantial investment. 

Measuring Fidelity of Program Implementation 

The ultimate goal of any demonstration project evaluation is to determine unequivocally what works 
best, for whom and under what conditions. The use of a true experiment for the Chrysalis evaluation 
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is a major step toward this goal. Another key ingredient is the adoption of uniform standards for 
program implementation. The issue of fidelity of the demonstration project implementation centers 
on a clear statement of program goals, establishing content validity in the curriculum and instruction, 
and defining what are acceptable (and unacceptable) levels of program variation. 

The authors see that these interests might best be served by more targeted applied research. We 
suspect that even the most prominent Chrysalis components, i.e., the support group sessions, may 
be of insufficient dosage to yield a significant effect. We believe the evaluation task is to show the 
efficacy of the full Chrysalis model, not determine the effectiveness of each component in isolation. 

Defining the key components of an innovation has long been a subject of study. The Concerns- 
Based Adoption Model (CBAM; Hall & Herd, 1987) offers a proven research methodology for 
studying the fidelity of program implementation. As part of a multi-dimensional research framework, 
the CBAM researchers defined a diagnostic dimension. Innovation Configurations, to describe the 
operational forms an innovation can take. Using an Innovation Configuration Checklist clarifies the 
components critical to the implementation and other related components on a continuum from “ideal 
to “acceptable” to “unacceptable.” 

These definitions are directly related to the fidelity of the implementation. Some level of variation 
is tolerated for critical components. This scaling of the key elements of the project will enable project 
staff to make more appropriate decisions about the use of program, monitor the implementation, 
refine promising practices, and evaluate program effects. At the present time, the project 
management team is piloting a Chrysalis Student Inventory \o further define the curriculum objectives 
in preparation for scaling the key elements of the program for students and case managers. 

Another issue closely related to program implementation is the interest of policymakers in knowing 
if particular subgroups of students benefit more from the intervention than others — African 
Americans, school dropout, or younger students versus older youth, etc. These are legitimate 
interests, but require disaggregating the project sample size (already meager due to lower enrollment 
as noted above) to levels possessing insufficient statistical power to detect effect sizes of interest. 

From the authors perspective, there is some incongruence between the project design recommended 
in the funding agency’s Request for Applications (RFA) and the instructions for the Annual Evaluation 
Report. The RFA suggested that “what works” in prevention are comprehensive, multi-faceted 
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interventions targeting multiple domains of individual, family, school, peers, and community. From 
the outset. Chrysalis was designed to provide just such an intervention in the lives of young women 
who had experienced child abuse. The instructions for this annual evaluation report, however, 
encouraged evaluators to break the intervention down into its components, relate specific outcomes 
to each, and report on the effectiveness of each component in achieving its intended outcomes. 

As described in Chapter 2 of this report. Chrysalis consists of six distinct intervention components, 
ranging from weekly support groups and ongoing case management support to a single, year-end 
celebration event. Program designers believe that these components will influence the young 
women not only in themselves, but also in combination with each other — a sort of interactive effect 
in which the whole is greater than the sum of its parts. Clearly, to modularize the Chrysalis 
intervention, separating its component parts and dealing with each as though it were an isolated 
intervention, is incongruent with the reality of the project’s implementation and probably its effects. 
The author’s ask for further consideration of the congruence between recommended project design 
and the evaluation as represented in the required outline and its instructions. 

The Chrysalis staff and evaluation team are trying to be as responsive as possible to the interests 
of all stakeholders in the project. We have updated information on concerns or requests expressed 
by the funding agency or their contractors in the body of this report according to their specifications. 
The authors appreciate the special interests of the federal perspective and trust that the funding 
agency reciprocates and values the extensive efforts that the Chrysalis project staff and research 
team have implemented to effect a state oUhe art quality evaluation of this program. 

Enhancements to Chrysalis Research and Evaluation 

Project Chrysalis features a rare combination of evaluation staffing — an evaluation professional 
within the Portland Public Schools and another from an independent research firm in the community, 
RMC Research Corporation. This unique collaboration places a premium on communication and 
clarity of roles and responsibilities for the evaluation as each staff brings specific strengths. 

In Year Two, the Chrysalis evaluation expanded into three new areas — a student drug use validity 
study, an HIV/AIDS Education Project, and a Media Literacy Project. These new programs brought 
an additional burden to the evaluation collaboration. From the evaluators perspective, there was an 
incongruence in the limitations set by the funding agency on the sharing of student information 
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between the professional evaluators. During the data collection and analysis for the special projects, 
there were many times when the evaluation would have been facilitated, and indeed strengthened, 
by sharing student names and project code numbers on a "need to know" basis between the internal 
and external evaluation agencies. Unfortunately, the validity study bore the brunt of this mandated 
segregation of'participant names and code numbers which were available to the internal evaluator, 
but not to the .external contractor. Due to these complications, the external contractor was forced 
to suspend the original research design for the Validity Study of Self-Reported Drug Use. While the 
study was eventually completed, it was only through the creative and resourceful approaches of all 
involved, and at the cost of extra days, that the participants names were held in confidence. 



The communication between internal and external evaluators and program staff has resulted in a 
project evaluation that offers maximum benefits from the perspective of all stakeholders. We believe 
it is a credit to the program management for their willingness to bear the increased costs of this 
staffing plan, and suggest that CSAP reflect on the merits of this strategy whenever it is available. 



Project Implementation 



The results reported in Chapter 4 focused primarily on examining the relationships among different 
health risk behavior areas. A few of the analyses also focused on relationships among protective 
factors. The results of these analyses are grouped within six topic areas: alcohol use, cigarette use, 
marijuana use, sexual activity, violent behavior, and suicide. The following highlights summarize the 
key findings from the outcome evaluation of the second year of the Chrysalis Project: 

Alcohol Use 

• Over the course of the 1995-96 school year, a higher percentage of Chrysalis 
girls reported using alcohol in the past 30 days, while the percentages remained 
essentially the same in the Research group. 

• In general, students who participated more in Chrysalis activities (high dosage) 
reported more negative attitudes toward ATOD use and stronger refusal skills. 

• Chrysalis students who attended the all-student year-end celebration event report 
more negative attitudes toward ATOD use. 

Marijuana Use 

• Over the course of the 1 995-96 school year, a higher percentage of Chrysalis 
girls reported using marijuana in the past 30 days, while the percentages 
remained essentially the same in the Research group. 

• Students at higher risk levels (8 or more risk factors at intake) reported higher 
use of marijuana at school. 

• Chrysalis students attending more support groups reported lower rates of 
marijuana use in the past 30 days. 
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• Research students receiving more case management sessions and hours were 
more likely to have used illicit drugs at or before school, and to have been in a 
car where the driver had recently used drugs. 

Cigarette Use 

• Students at higher risk levels (8 or more risk factors at intake) reported higher 
use of cigarettes at school. 

. • Students in both groups who received more case management sessions and 
hours report parents and friends with more positive attitudes toward ATOD use. 

Sexual Activity 

• Over the course of 1995-96 school year, more students in the Chrysalis Program 
and Research group reported having had sexual intercourse. 

• Over the course of the year, fewer Chrysalis Program group students became 
pregnant than Research students. 

• Older students reported having their first sexual intercourse at a younger age and 
were more likely to report having been sexually abused. 

• Students at higher risk levels (8 or more risk factors at intake) reported higher 
rates of sexual intercourse with more partners. 

• Chrysalis students attending more support groups reported fewer instances of 
sexual intercourse in the past 3 months. 

• Chrysalis students attending more HIV/AIDS sessions report lower self-efficacy. 

Violent Behavior 

• Over the school year, fewer students in both groups engaged in physical fighting. 

• While the main effects of fighting go down throughout the high school years, 
Chrysalis students had bigger declines in fighting than would be expected. 

• Chrysalis students receiving more case management sessions and hours were 
more likely to have carried a weapon to school in the past 30 days; and more 
likely to have lower GPA and school attendance. 

Suicide Risk 

• Older students reported more suicide plans and attempts at suicide. 

• Students at higher risk levels (8 or more risk factors at intake) reported higher 
rates of considering, planning, and attempting suicide. 

-> • Research students receiving more case management sessions and hours were 
less likely to have attempted suicide in the past year. 

Resiliency Outcomes 

• Chrysalis students at higher levels of risk report being less assertive. 

• Over the course of the year, students in both groups report stronger bonding to 
family and a stronger sense of personal competence. 

• Older students in both groups are more assertive and have a stronger sense of 
personal competence. 

• Over the course of the year, younger students report a decrease, while older 
students report an increase, in positive outlook toward the future. 

• Chrysalis students report less of an increase in prosocial bonding to family and 
school over the year than do Research students. 

• Research students receiving more case management sessions and hours feel 
less personally competent, have a less positive outlook on the future, feel less 
guidance and support from adults, and less assertive with their peers. 
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Summary 

The data presented in this report on the second year of the project offer a striking portrait of the 
Chrysalis high risk female adolescent population. Despite lower enrollment than anticipated, staff 
changes, and other limitations, the project has shown remarkable resiliency and maintained steady 
progress toward identifying problems and framing solutions. While recognizing this success, the 
authors recommend continued attention be given to schools in which the project has not been fully 
implemented, so that any obstacles to implementation can be overcome. 

In this report, several analyses were done to examine relationships between the various health risk 
behaviors of participants and program outcomes. Other analyses detailed differences in responses 
between Program and Research students on protective factors. Clearly, the results show that 
relationships exist between young women with a history of physical, sexual, and emotional abuse 
and increased use of alcohol and other drugs, sexual behaviors, violence-related behaviors, and 
suicide ideation and attempt. Some of the reported results are not surprising and corroborate those 
presented in other studies, v^rhile the meaning of other findings remains unclear. The research team 
and project staff will continue their animated discussions and next level analyses to discover the 
meaning and reasons behind the data. As of this writing, focus groups with students are planned 
to investigate the issues of survey honesty and students' perceptions of program risk. Thus, the 
information contained in this report will be immediately useful for those involved in implementing the 
Chrysalis program. 

In summary, as the literature suggests, no single approach will work for all students. The Chrysalis 
demonstration project has offered a variety of options and support services to address multiple 
health risk behaviors associated with the high risk female adolescent population. Project staff report 
that the success of the program to date lies in the involvement of caring, well-informed, well-trained 
personnel. The staff have uncovered extraordinary learnings about implementating a project with 
this target population. Key among these learnings is the impact of dosage — the more project 
activities we get students to attend, the more positive are the outcomes. Chrysalis is a place where 
students can develop competence and confidence in their abilities, where they can have successful 
experiences and learn constructive behaviors, where they can find a passage to a safe future. We 
anticipate that with each passing year there will be increasing evidence of the positive impact the 
program has on the lives of students. As the research proceeds into the third year. Chrysalis 
embodies these lessons learned and continues to build on the resiliency framework. 
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APPENDIX A 

Support Group Curriculum Outline 



Chrysalis Support Group Curriculum 
Table of Contents 



General Information 

Chrysalis Intake Process 
Intake Form 

PPS Child Abuse Reporting Procedures and Resource Materials, 1995-96 

Session Format 

Session Worksheets 

PPS School Calendar, 1995-96 

Curriculum: List of 1 995-96 Topics (topics may involve more than one session) 



Topic 1: 


Who Are We? What Is A Chrysalis? 

Chrysalis Fact Sheet, Chrysalis Group Agreements, Sample Shield 


Topic 2: 


What Is Abuse? 
Examples for Brainstorm 


Topic 3: 


Storytelling 

Journalling General Guidelines 
Why Tell? 


Topic 4: 


Secrets/T rust 
Family Loyalty 

There’s No Such Thing as a Dragon 
Secrets 


Topic 5: 


Storytelling Through the Family 

Examples of Happenings That Broke My Heart 

When 1 Was Growing Up 

Moms 


Topic 6: 


Trauma and Its Effects 
The Effects of Trauma — ^A Model 
How Survivors Have Been Impacted 
How Hurtful Experiences Impacted On Us 
Contaminated Thinking Card Exercise 
Truths/Right Thinking/Right Beliefs 
Wrongs/Wrong Thinking/Wrong Beliefs 
Being Hurt and Hurting Others 
S.O.S. ... Be On the Lookout 


Topic 7: 


Shame/Blame/Depression 
Shame and Guilt 
It’s Not Your Fault 

The Depression Continuum & Depression (Information for Leaders) 
Features of Different Forms of Depression 
Criteria for Major Depressive Episode 
Guilt 

The Ugly Girl 

Where Does Our Shame Come From? 

Negative Rules and Negative Messages Commonly Heard 
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Topic 8: Boundaries 



Establishing and Communicating Boundaries 
Boundary Exercise 



Locus of Control Questionnaire & Key to LOC Questionnaire 
Values Exercise 

Autobiography in Five Short Chapters 



Topic 10: How We See Ourselves 

Image Bags Exercise & Body Sizes and Differences 

Topic 11: Intimacy 



Intimacy Brainstorm Exercise & Intimacy Scale 
Survivors’ Beliefs About Relationships 
Girls Empowerment (Love Is, Love Isn’t) 
Danger Ahead! 

The Mathematics of Battering 
Relationship Contract 



Learning to Love 
Sex Miseducation 
Hellbent on Redemption 
Fast talk 

Sexual Effects Inventory 

Hooks and Pitfalls in Relationships 

Abuse in Relationships 

Discovering Your Triggers 

How Do I Love Thee; Let Me Count the Ways 

Analysis for How Do I Love Thee 



Women’s Health Bill of Rights 
Pandora and Me 

The Stereotypic Feminine Woman 
Power 

Someday My Prince Will Come (And Other Lies Snow White Told Us) 

Sexual Assault 

How Things Have Changed 

For Strong Women 



Handouts/Notes 

Open Sessions Suggested Topics 
Feelings and Feeling States 
Learning to Relax 
Me As Mom (Teen Parents) 

Values Survey 

Sexual Rights Questionnaire 

Group Roles — Unhealthy to Healthy Scenarios 

Problem Solving Exercise 

Survival Kit 



Topic 9: Powerlessness 



Topic 12: Sex and Relationships 



Topic 13: A Celebration of Women 
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The Validity of Self-Reported Drug Use 



Purpose of the Study 

The Chrysalis project received funding from the federal Center for Substance Abuse Prevention to 
conduct a special study to validate the self-report of certain health risk behaviors on the project's 
Risk and Resiliency Index survey. The study features the use of hair specimen analysis as a 
validating measure of students' self-reports of their illicit drug use. As always, results of the hair 
analysis and the self report survey were kept completely confidential. 

Need to Validate Self-reports 

Surveys of alcohol and other drug (AOD) use are often criticized by policymakers and the general 
public as relying too heavily on the honesty of the respondent. The National Institute on Drug Abuse 
(NIDA) has identified other sources of error that can be equally biasing O’ urner et al, 1992) in the 
collection of this information via self-report. Young people's reading levels, comprehension of 
complex concepts within drug use items, and attention spans — in what are often very long surveys 
— are other complicating factors having nothing to do with their willingness to disclose or respond 
honestly to these questions. 

Project Chrysalis aspires to reduce AOD use behaviors among its participating young women, and 
this information was collected via a self-report survey. To lend confidence to the interpretation of 
these results, a validation study was designed and implemented by project evaluation staff. 

Literature on Validity of Self-Reported AOD Use 

A review of self-report validation studies by Harrison (1995) shows that the validity of self-reporting 
of drug use is still in need of further research. Most validation studies have been conducted on the 
criminal justice population who have higher rates of drug use than the general population, thus 
making it difficult to generalize results from these studies to the general public. Of the validation 
studies that have been done, it was found that the validity of self-reports is maximized when the 
validation method is done close to the drug event; the drugs self-reported are socially acceptable; 
and when the self-report data are collected under controlled conditions. 
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The validation method used most often has been urine testing. Studies have sho\wn that there is a 
high degree of congruence behween self-report and urinalysis; but there are limitations to this 
method, most notably the short windo\w of time available to capture the drug event. Detection of illicit 
drugs through urinalysis has a limited time frame of between two to three days after ingestion, with 
the exception of marijuana which may be detected up to four weeks. 

Hair analysis is a relatively new method for validation of self-reports, but it has not had the 
widespread acceptance from the scientific community as has urinalysis. It has obvious advantages, 
including ease of specimen collection and a much longer window of detectability. (See next section 
for further discussion on pros and cons of hair analysis). 

Less intrusive validation methods have been employed including interviewer administered drug use 
surveys and self-administered surveys. The self-administered surveys have the advantage of 
providing a high degree of confidentiality to the respondent, an important factor in the validity of their 
answers. Typically, higher rates of drug use are reported using a.self-administered answer sheet 
as opposed to answering verbally to an interviewer. 

According to Harrison (1995), even though the validation methods are flawed to some degree, the 
research that has been done so far shows that self-reports provide reasonably valid estimates of 
drug use prevalence. Lifetime drug use in self-report surveys appear to be the most valid, followed 
by past year and then past month use. 

Pros and Cons of Hair Sampling 

Those who have done research on the feasibility and accuracy of hair analysis believe it is 
convenient, safe and effective (Baumgartner & Hill, 1992; Mieczkowski, 1992). Two of the most 
important advantages of using hair analysis to detect the presence of illicit drugs is the long time 
span or window of detection it provides and the potential for estimating the ingested dosage levels 
of illicit drugs based on the test results. Depending on the length of hair, the presence of drugs after 
ingestion can be detected for several months and possibly for years. According to Mieczkowski 
(1992), detection of illegal drugs can be achieved at sensitivity levels comparable to those of 
urinalysis. 

Hair analysis and urine analysis offer complementary time frames. It takes approximately five to 
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seven days before an illicit drug that was ingested is detectable in the hair shaft while ingested drugs 
appear rapidly in urine. Hair analysis can be used to examine a large time span which provides the 
long-term history of an individual's drug use. 



Mieczkowski (1992) states that the basic scientific analytical process involved in hair analysis 
technique is well established. The controversy surrounding hair analysis lies in the interpretation of 
hair analysis outcomes and how these outcomes may or may not be utilized. Drug testing done for 
screening purposes, as many employers are now requiring of their employees, has legal implications 
as to its acceptability as evidence in a court of law. The controversy lies in whether hair analysis 
is sufficiently accurate and reliable, although there have been a number of cases where results of 
hair analysis has been used in forensic cases and allowed as evidence (Psychemedics, 1994). 



Another concern is that the processes by which illicit drugs or other substances are absorbed into 
the hair are not well understood, making some researchers uneasy about interpreting hair test 
results. For example, passive exposure to drugs (i.e. living in an environment where others smoke 
marijuana but the individual does not) may result in a positive test. Proponents tor hair testing say 
that the problem of environmental contamination can be addressed by careful analysis of specimen 
washing and other techniques. 




Mieczkowski, well-known tor a number of research studies on the reliability of hair analysis, states 
that just because there is controversy about hair analysis, doesn't mean that it should stop being 
used as a form of drug testing. Urinalysis was met with the same kind of controversy and resistance 
by researchers, but now it is widely accepted and used. 



Mieczkowski believes that there is a need tor large scale hair testing on select populations in order 
to provide more empirical evidence tor the reliability of hair analysis tor drug detection. More studies 
will then be able to address and answer those problems of the method expressed by those in the 
research field. 




Sensitivity of Hair Sampling 

Research has shown (Mieczkowski, 1992, 1995) that hair testing otters advantages over urine 
testing tor detection of illicit drugs. Drugs and drug metabolites remain isolated in the hair shaft 
indefinitely, thus providing detection during a much longer period of time (approximately 60 days of 
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use can be seen in one inch of hair) than drug levels in urine (three days of use can be seen in a 
urine specimen and the specimen can be kept for a very limited time). 

It is estimated that it takes about five to ten days before an illicit drug that was ingested is detectable 
in the hair shaft. A standard hair test covers a period of approximately 90 days. According to 
Mieczkowski (1992), significant amounts of material that comes in contact with the hair through 
external means, such as through passive smoke, can be removed by appropriate washing and 
measurement of the wash outcomes. This is confirmed by Psychemedics Corporation, the company 
that did the testing for this project who use a number of special processes for handling hair samples 
that have been chemically treated or exposed to other external contaminants. Psychemedics states 
that there has been no evidence that passive inhalation of marijuana can produce positive hair test 
results. 

All hair samples collected for this project were tested by Psychemedics Corporation for marijuana, 
cocaine, opiates, methamphetamine and phencyclidine (PCP). Cutoff levels are the concentration 
levels above which the hair specimen is deemed to be positive and below which it is deemed 
negative for a particular drug. Exhibit 1 summarizes the drugs and the cutoff levels for the 
substances tested by Psychemedics Corporation. Although the amounts of drug deposited in hair 
are small, by analyzing 10 mg of hair the laboratory can measure the amount of drug deposited by 
even low level drug users. Except for marijuana, this amount is approximately correlated to the 
severity of drug use. Hair cutoff values represent minimum concentrations for the detection of low 
level chronic drug use. Samples reported as positive show the presence of drug in two independent 
analyses. This two-step analysis is a strong protection against false positive reports. 



Exhibit 1 

Validity Study Substances and Cutoff Levels 







Cocaine & Benzoylecognine (Metabolite) 


5 nanograms/10 milligrams of hair 


Methamphetamine 


5 nanograms/10 milligrams of hair 


Opiates (Codeine & Morphine) 


5 nanograms/10 milligrams of hair 


PCP 


3 nanograms/10 milligrams of hair 


Marijuana Metabolites 


10 picograms/10 milligrams of hair 



Note: Five nanograms is equal to one-fourth gram. Ten picograms is equal to one-tenth of a nanogram. 
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Method 

Working with Case Managers 

The validity study coordinator attended a case manager's meeting in April 1996, to inform the 
Chrysalis case managers about the validity study and their role in that study. At the meeting, they 
were given a question and answer fact sheet about the hair collection process and the drug test. 
They were asked to inform their Program and Research students about the validity study a week or 
two prior to the administration of the survey. The case managers answered students' questions at 
that time, or they told them that they could ask the validity study coordinator questions at the time 
of the survey administration. In order to follow the initial sampling design, the case managers were 
instructed only to say to the Program and Research groups that some of them will be asked to 
participate in this study and some will not. The case managers did not know which girls would be 
asked to participate until the day of the survey administration. 

The validity study coordinator contacted the case managers to confirm the survey administration 
dates at their schools. At least two visits to each of the 12 high schools were made by PPS 
evaluation staff and the validity study coordinator during April and May. Ideally, it would have made 
the process easier if the survey was administered to the maximum amount of participants at one 
time. In other words, a case manager would administer it to the Program girls at one time and to the 
Research girls at another time with possible follow up time scheduled for girls who were absent on 
those dates. On a few occasions, the validity study coordinator was able to schedule follow-up hair 
collection times, but there were too many absences at the time of the initial survey administration 
and hair collection to make it feasible to schedule as many follow-up sessions needed. 

Sampiing Design 

All Chrysalis Research and Program participants were randomly assigned to one of three groups 
(see chart below). When receiving their Risk and Resiiiency survey, participants assigned to Groups 
1 and 2 also received a consent form that asked them to participate in the validity study that involved 
a drug test using a sample of their hair. If they consented, they had the test done after they 
completed the survey. When turning in their surveys, the girls assigned to Group 2 were informed 
that they were not going to be tested after all. When turning in their surveys, the girls assigned to 
group three then were asked if they wanted to participate in the study and were asked to sign a 
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consent form and wait to be tested along with the other girls from Group 1 . The case manager and 
PPS evaluation staff were asked monitor this process. 

The initial study design is shown below in Exhibit 2, along with the number of students randomly 
assigned to each group. 



Exhibit 2 

Research Design for the Validity Study 











Program participants 


49 


48 


44 


Research participants 


49 


57 


37 



Group 1 : When receiving the survey, they also received a consent form. When they turned in their 
survey, they went to have the hair collection sample done. 

Group 2: When receiving the survey, they also received a consent form, but at the time of turning 
in their survey, they were informed that they would not be tested after all. 

Group 3: Upon turning in their survey, they were asked to participate in the validity study and were 
given a consent form at that time and then proceeded as Group 1 . 

The rationale for using three groups in the study involved interest in testing whether advance notice 
of the hair sampling influenced students' responses to the survey. Research indicates that often the 
mere suggestion of such a procedure (as in Group 2) can have the desired effect. Similarly, the 
absence of advance notice (as in Group 3) may have the opposite effect. 

The rationale for including the Research group in the study is to test whether there is greater or less 
validity in the responses of the two groups. There is research that suggests that students who are 
participating in a program that encourages open communication and self disclosure will tend to 
respond more honestly to questions of a somewhat personal nature, such as those on the Risk and 
Resiliency Index. 

Hair Sampling Technique 

The validity study coordinator had a list of Chrysalis participants' ID numbers. This list was only 
seen by the coordinator and PPS evaluation staff and used for the purposes of the study. On the 
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date of the administration of the survey, the validity study coordinator verified with PPS evaluation 
staff the girls to be tested. PPS evaluation staff then checked to see if the girls were in attendance 
and made sure consent forms were signed. 

The hair collection for the drug test was conducted according to a strict protocol and guidelines as 
outlined by Psychemedics — a California-based company that was responsible for doing the 
laboratory tests on hair samples. The hair testing company provided the necessary equipment for 
the collection of hair samples (scissors, hair clips, forms and hair sample collection pouches). A 
separate room was used to conduct the hair collection. The girls were sent in one at a time by the 
case manager. After the hair collection, the girls returned to their regularly scheduled classes. 

The participants entered the room and were asked to be seated. The validity coordinator explained 
the hair collection procedure to each participant before beginning the process. The procedure 
involved standing behind the subject in order to take a hair sample from the "crown" of the head. 
A hair clip was sometimes used to hold hair out of the way. If the hair was fine or thin or very short, 
a sample from more than one area of the head was taken. A small lock of hair, visibly equal to one- 
half inch wide when held flat across the finger, was cut as close to the scalp as possible. The hair 
sample was then placed into a strip of foil following defined procedures. This foil was placed inside 
a card that was sealed, signed and dated by the validity coordinator. The card was placed into a 
plastic pouch which had an adhesive closing to protect the integrity of the hair sample until it reached 
its testing destination. The hair collection process took approximately three minutes to complete. 
The samples were kept in a locked cabinet until sent via Federal Express to the hair testing site. 

Results 

Erosion of Sampling Design 

The initial sampling design was changed about half way through the survey/hair collection process. 
Because of absences during survey administration, attrition of Chrysalis participants, and a higher- 
than-anticipated refusal rate, the number of hair samples collected was discouragingly small. To 
increase the number of samples, it was decided by Chrysalis staff and the evaluation team, to open 
up the hair collection process to all three groups. Following this point, all girls were asked at the time 
of survey administration if they wished to participate in the validity study. 
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At the year-end retreat for Program girls, an area was set aside by the validity study coordinator to 
gather hair samples from girls who had not had the opportunity to participate because they were 
absent or they were in the group that was not asked initially to participate or in the event that they 
had changed their mind from when first asked. Case mangers and Chrysalis staff spoke to the girls 
individually during the course of the retreat to see if they wanted to participate in the validity study. 
This method also increased the number of hair samples. 

In addition, girls who missed the initial survey administration because of absence or other reasons 
were mailed the survey along with the validity study consent form so they were informed of the study 
and could participate if they wished. In all, 74 surveys and consent forms were mailed out (36 
Chrysalis, 38 Research students). Of these, only six students agreed to participate in the validity 
study (4 Chrysalis, 2 Research). One girl who was mailed the survey/consent form was at the retreat 
and chose to participate in the study. 

Although these mid-course corrections resulted in a sample size large enough to conduct statistical 
tests on the results with reasonable power, the abandonment of the original three-group design 
precluded the unambiguous test of many of the associated hypotheses. These will be classified in 
the subsequent discussion of the results. 

Consenting vs. Non-consenting Students 

Since participation in the validity study was voluntary, it is important to examine characteristics of 
the consenting students to determine whether they are generally representative of project 
participants as a whole, or are a more “special” group. To the extent that participating girls are 
different from the project students overall, results of this validity investigation may not be 
generalizable to the population of young women these project participants represent. Secondarily, 
if there is more or less representativeness among Chrysalis students than Research group students, 
it suggests that these results be analyzed separately. 

An initial comparison of interest is simply in the relative percentages of Chrysalis versus Research 
group students who consented to participate in the study. Of the 92 Chrysalis students who were 
asked to participate, just over half (53%) agreed; while, of the 96 Research students who were 
asked, less than one-third (31%) agreed to participate. These numbers do not include the 74 
students who were asked in the follow-up mailout; nor do they include students who were first asked 
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for their consent, but after completing their surveys were told they would not need to contribute a hair 
sample after all (i.e., Group 2 in the initial design). 

The differences in these proportions likely reflects the greater willingness on the part of Chrysalis 
program students to participate in project-related activities. Case managers approached Research 
group students with the same protocol for participation in the validity study, but the lower level of 
investment in such activities is not surprising. In and of themselves, these differences in 
participation rate do not constitute a bias. That is, the 31 percent of Research group students may 
be an equally representative sample of all of their group as is the 53 percent of Chrysalis students. 
To address this, we must look at the characteristics of the students included in both groups. 

A primary concern in evaluating the representativeness of the consenting sample, as noted above, 
is in whether students who actively use the substances detected by the hair analysis procedure 
would be less likely to agree to participate in the study. Secondarily, any differences in prevalence 
of health risk behaviors between consenting and non-consenting students could threaten the 
generalizability of the findings. 

Two health risk behaviors — self-reported marijuana use and alcohol use in the past 30 days — 
were explored to address this representativeness issue. Prevalence rates of Chrysalis and 
Research students who participated in the validity study are contrasted with those rates for the 
students who refused to participate in the study in Exhibit 3 below. Comparing Chrysalis vs. 
Research students who did and did not consent to participate tells us whether any bias in the 
participating sample is essentially the same for the two groups. That is, if heavier AOD users tend 
to refuse participation in the Research group, is this also true of the Chrysalis group? If one believes 
that participation in the Chrysalis project lends itself to more cooperation in any related activities (as 
alluded to above), it may be that this decision is made independently by Chrysalis girls, but Research 
girls give greater consideration to possible consequences or are just generally less interested. 

Also included in Exhibit 3 are the prevalence rates for all Chrysalis and Research students who were 
included in the Spring posttest for the overall evaluation of the project. These are included to afford 
the comparison of participation samples with the total groups of Chrysalis and Research group 
students involved in the project. While the first comparison relates to the equivalence of the 
Chrysalis and Research samples participating in the validity study, this one assesses the overall 
representativeness of the participating groups to all those included in the Chrysalis project. 
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Exhibits 

Prevalence of 30 Day Alcohol and Marijuana Use 
Consenting and Non-Consenting Students 
and All Project Students 











Consenting (N=79) 


46.9% 


40.0% 


32.7% 23.3% 


Non-Consenting (N=109) 


44.2% 


42.4% 


39.5% 31.8% 


All Project Students (N=223) 


52.2% 


41.2% 


40.2% 25.5% 



Differences between consenting and non-consenting students in both Chrysalis and Research 
groups are non-significant for alcohol use, but more pronounced for marijuana use. In both groups, 
those students who consented to participate report lower prevalence of marijuana use than those 
who refused — a nearly seven percent difference among Chrysalis students and an eight percent 
difference among Research group students. This supports the experimental concern that students 
who use marijuana are less likely to participate in the validity study, but the reluctance is about the 
same in the two groups. 

While Chrysalis and Research group students share this difference between those consenting and 
refusing to participate in the validity study, their representativeness to the total group shows a 
different pattern. The higher rates of marijuana use among Chrysalis students as a whole (42.7%) 
than those that participated in the validity study (32.7%) suggests that they are a lower-using group 
of girls than the entire Chrysalis group. Results for validity participants and the total project group 
for Research students are more comparable. 

These findings suggest some caution in generalizing the results of the validity investigation to this 
target group as a whole, but confidence in the comparisons between Chrysalis and Research group 
students. 

Validating Self-Reported Illicit Drug Use 

Of the illicit substances included on the Risk and Resiliency Index, the hair sampling procedure could 
sensitively detect use of marijuana, inhalants, cocaine, amphetamines and heroin. Among those 
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students who participated in the validity study, self-reported prevalence rates of all substances but 
marijuana were so low (less than 5% per group, indicating one or two students using) that an 
assessment of the agreement between the hair analysis and self-report would be highly positively 
biased. That is, when marginal usage is this low, any sort of statistical index of agreement will 
indicate high agreement, due to the preponderance of negative results. 

With this in mind, the validity results reported here will focus on marijuana use. In all, 94 young 
women participated in the hair analysis process. Of these, 7 submitted hair samples that were too 
small to adequately analyze, and another 3 students did not respond to the 30-day marijuana use 
item on the survey. This left 84 Chrysalis and Research students for whom both a valid hair analysis 
result and a response to this survey item was obtained. Results of the initial comparison of these 
data are shown in Exhibit 4a below, 



Exhibit 4a 

Initial Results of Hair Analysis and Self-Reported Marijuana Use 









Physiological Test of Positive 


3 2 


5 


Marijuana Use (Hair Analysis) 

Negative 


57 22 


79 


Total 


60 24 


84 



Although the results of this comparison are cast in a very simple table, the interpretation can become 
quite complex. The agreement of the two assessment procedures (self-report and hair analysis) is 
represented by two cells — those students for whom both assessments found to be users (n=2), and 
those for whom both found to be non-users (n=57). Combining these two, there is just over 70 
percent agreement between physiological and self-reported assessments of marijuana use. 

Another approach is to focus on the “user” samples from each procedure. Cf the 24 students who 
reported some use of marijuana in the past 30 days on the survey, only two of them also were 
detected as positive by the hair analysis. This can be represented as an approximately 8 percent 
agreement rate. Conversely, of the five students identified positively by the hair analysis, two 
indicated they had used marijuana in the past 30 days on the survey. This may be expressed as 
a 40 percent agreement. 



Page 123 



Chrysalis Year 2 Evaluation Report ■ Appendix 



One of the circularities operating within a validity study of this sort is that neither one of the 
assessment procedures necessarily provides the Irue” picture of whether a student has used 
marijuana or not. Hair testing, as noted in our earlier review of the literature, is a promising 
technique, but it is not flawless. Self-report surveys, while carefully controlled in the Chrysalis 
evaluation, are also often questioned. Among other things, this makes it problematic to choose from 
among the agreement indices described above. That is, if one could be considered the “correct” 
procedure, it would suggest we calculate the percentage of times the other assessment results agree 
with it. 

Even with this uncertainty, and options for expressing agreement, the overall correspondence 
between the two procedures is not encouraging. Looking beyond their agreement, and interpreting 
the marginal frequencies of marijuana use from the two procedures suggests one possible 
explanation. Of those students included in this analysis, over 28 percent (24 of 84) indicated they 
had used marijuana on the survey; while the hair analysis procedure detected only about six percent 
— less than one-fourth of the self-report total. This discrepancy indicates that, either the girls were 
over-reporting on the survey or the physiological procedure is too conservative. The first explanation 
is unlikely in that this prevalence rate is comparable to other survey results obtained from other 
samples in other locales. To explore the second, the RMC Research team consulted with 
Psychemedics to see if it was reasonable to lower the detection threshold on the hair analysis to 
determine whether an appreciable portion of the self-reported users who had tested negative on the 
initial test would test positive with a lower quantity of marijuana evidenced in their hair sample. 



Results of this secondary analysis yielded an additional nine students who tested positively at this 
lower level, and another five who “just missed” at this threshold. Adding the nine to the “positive” 
category brings the agreement rate among self-reported marijuana users to just under fifty percent. 
Adding the additional five who “just missed” brings the agreement to about two-thirds. Lending their 
experience to this, Psychemedics staff were clearly more comfortable with including the positives 
from the lower threshold than they were with the “just missed” category. 



Applying this adjustment to the previous Exhibit 4a reconfigures the table as follows: 
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Exhibit 4b 

Results of Hair Analysis and Self-Reported 
Marijuana Use After Lowering Threshold of Hair Analysis Test for Students 
Who Had Reported “Any Use” on the Self-Report Survey 











' s V sv V ' ^ ^ 


Physiological Test of Marijuana 
Use (Hair Analysis) 


Positive 


3 


11 


14 


Negative 


57 


13 


70 


Total 


60 


24 


84 



While these results show improved agreement between the two assessment procedures, they are 
still not terribly impressive from a statistical point of view. Since the hair samples from the self- 
reported non-users were not retested using the lower threshold, we cannot compute overall 
agreement indices with any certainty. At best (i.e., if all self-reported non-users still tested negative), 
the overall agreement index would be 81 percent — comparable to the 79 percent agreement 
between urinalysis eind self-reported marijueina use in a criminal justice population (National Institute 
of Justice, 1991), and better than many found by Mieczkowski (1990) in his comprehensive review 
of self-report validation studies. This appears to be a high figure, but one that still indicates 
approximately one in five students have used marijuana according to one method, but have not used 
according to the other. 

In conclusion, we remind ourselves that this was a highly experimental procedure, using a relatively 
non-intrusive physiological measure to validate self-reported marijuana use. The literature suggests 
promise for hair analysis, although it is not yet recognized by the National Institute on Drug Abuse 
(NIDA) as a valid technique. Other researchers (Harrison, 1995) have pointed out that it is highly 
preferred to urinalysis due not only to its relatively non-intrusive nature, but also because its window 
of detection of drug use is so much wider. Given that the secondary analysis results here could yield 
agreement rates comparable to those found in urinalysis studies, hair analysis may be a reasonable 
method for use in a school population. 

We suspect that there is still much to learn in the implementation of such a procedure, however. Our 
consultations with the professionals at Psychemedics indicated to us that there are still many key 
decisions to make in setting standards and criteria for the physiological testing, particularly when 
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working with this population. It is reminiscent of the decisions the applied researcher makes in 
setting acceptable probabilities for Type I (false positive) and Type II (false negative) errors. 

We close the report with further reflections on lessons learned in the implementation of this 
procedure in the context of the Chrysalis project, its staff | and population of participants. 

Discussion 

Implementation Difficulties 

Some of the difficulties involved in the validity study process revolved around confidentiality issues. 
Access to Chrysalis Program and Research participants' names was not open by RMC Research 
staff, according to Human Subjects Protection policies mandated by CSAP. This lack of access 
impeded the hair collection process, but it especially compromised any chance to do follow-up. RMC 
Research staff had access to participants ID numbers but not their names, while case managers had 
access to participants names, but not ID numbers. If the validity coordinator (RMC staff) wanted to 
do any follow-up for absent Program or Research girls, she could not do it without instituting a long 
and complicated procedure involving PPS evaluation staff and the case managers. Specifically, the 
validity coordinator would have to call the PPS evaluation staff, giving the ID numbers for absent girls 
for each school and group, then the PPS evaluation staff would have to call each case manager and 
tell her who those girls were so they could contact them and arrange a time for them to come in for 
the study. Then the validity coordinator would have to call the case manger to see when the girl 
could have the sample done but could not ask for the girl’s name. The logistics of doing this for 
every absent girl were simply not feasible. Program management and RMC evaluation staff agreed 
that to pursue such a process would strain the cooperative relationship among program and 
evaluation staff that had been growing so positively over the initial two years of the project. 

Another difficulty with the validity study was the large time commitment that was necessary that was 
made by the validity study coordinator in order to arrange for and complete the hair collection 
process. Not only was it time consuming to go to each high school, but the coordinator would wait 
at each school for 20 to 40 minutes before the girls were finished taking the survey and were ready 
for the hair collection process. Sometimes there were no girls who would choose to participate and 
so the coordinator's time was not put to effective use. Time was also needed for preparation to get 
the various hair collection items together and organized; forms and envelopes had to be filled out 
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with the validity coordinator's name, signature and other pertinent information about each testee. 
Girls' Attitudes about Participating 

The universal concern of the girls that chose not to participate in the validity study was that the 
validity coordinator would cut too much of their hair. This was a concern for the girls who did 
participate, but they were willing to go ahead with the procedure with assurances from the 
coordinator that only a small amount of their hair would be taken and it would not be noticeable. No 
other concerns were expressed by the girls. 

A few of the girls wanted to know if they could get the results of the test for their own knowledge. 
One girl wanted the results so in case she got a job with an employer who required drug testing, she 
could show the employer the results of this test. 

Anecdotal Evidence of Use 

Several times during the validity study process, study participants told the validity coordinator that 
they had recently smoked marijuana the night before or several days before the hair collection day. 
The girls wondered if their recent use would show up in the hair test sample, but they were not 
concerned if the results were positive. The hair test results were not to be used for any other 
purposes than the study, and participants knew this, but it was interesting to note that these girls felt 
free to tell the validity coordinator about their drug use. 

Chrysalis project staff reported similar revelations by the girls when they discussed their participation 
in the validity study. One girl wanted to know if her hair test would show positive results because 
she had recently been "hotboxing." (Hotboxing is riding in a car with people who are smoking 
marijuana with the windows rolled up). The girl said she didn't smoke marijuana but wondered if 
being exposed to the second-hand smoke would give a positive reading on her hair sample. 

Only five hair samples of the 94 taken for the study showed positive for marijuana use. With the 
numerous anecdotal accounts of the girls use and the larger number of positive responses reported 
on the Risk and Resiliency Index, it seemed that there would be more positives than what were 
reported in the hair analysis. 
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Our View of the Utility of Hair Sample Technique in School/Educational Settings 

Hair analysis was used over 30 years ago to test for the presence of toxic substances in forensic 
settings and to assess exposure to toxic metals in at-risk populations (Harrison, 1995; Mieczkowski, 
1992). And it has been used since the 1970s in detecting the presence of illicit drugs and in 
validating self-reported drug use. 

As stated earlier in this report, hair analysis has been used on adult populations who have been in 
the criminal justice system or who have been tested at their place of employment. Psychemedics 
Corporation indicated to us that the current study was their first experience using hair analysis to 
validate self-reports of drug use for a high school population. 

Implementing a hair testing study in a school-based setting had its challenges including establishing 
confidentiality procedures and obtaining cooperation from enough girls to have a useful sample. It 
may be that both the participants and the case managers were not adequately informed and/or did 
not understand the reasons for the study. 

It might be feasible to replicate a self-report validity study using hair analysis for other at-risk 
populations, but the procedures would need to be carefully looked at and refined and implementation 
time would need to be increased. 

In retrospect, a replication of this validity study would (a) spend more time explaining the importance 
of the study to case managers in the hopes this would improve student participation; (b) involve more 
research staff in carrying out the logistically demanding process of data collection; and (c) insist on 
access to students' names (confidentiality still assured) so that followup procedures could be 
implemented and the integrity of the initial design be maintained. 

With all of these caveats, RMC staff view this method of validation as having promise in a school- 
based setting. Secondary analysis of the samples, as reported earlier, yielded more useful results; 
and implementation lessons learned, if acted upon, could make the conduct of such a study more 
feasible. 
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Evaluation of Supplementary Grant: 

HIV/AIDS Education Project 
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Evaluation of Chrysalis HIV/AIDS Education Project 



Purpose of the Project 

The prominent role schools play in providing effective HIV/AIDS prevention education is well 
documented in research literature and by state and national surveys. According to the 1995 Oregon 
Youth Risk Behavior Survey (Oregon Health Trends, 1996), a majority of students identified school 
as their primary source for HIV/AIDS information and believed that increased knowledge about 
HIV/AIDS had a positive impact in reducing high-risk behaviors. Public schools in Oregon are 
mandated by law to incorporate HIV/AIDS instruction into the curriculum at all grade levels. Portland 
Public Schools adopted a policy statement on HIV/AIDS in 1988 and the topic is covered as part of 
the comprehensive and age-appropriate education about sexuality provided by the district. Project 
Chrysalis demonstrates the key role that schools can play in coordinating comprehensive services 
for high-risk student populations. As part of these comprehensive services, the project provides an 
excellent opportunity for further emphasizing this important health issue to program participants who 
constitute a particularly vulnerable population. 

Approximately one in 250 people in the United States is infected with the HIV virus. HIV/AIDS is 
spreading rapidly among teenagers and young children, thus identifying teenagers as one of the 
populations most at risk for contracting the virus. Typical adolescent attitudes of invincibility and the 
activities in which many teenagers engage dramatically increase their risk for acquiring HIV/AIDS. 
Since the Chrysalis program participants are a population vulnerable to risky sexual behavior, it is 
important that the serious nature of HIV/AIDS be underscored by giving them extra resources and 
information. The HIV/AIDS curriculum supplement is a way that this population has received the 
extra emphasis regarding this topic. 

The research literature cjtes a broad array of harmful effects resulting from childhood sexual, 
physical, and emotional abuse and neglect (Downs, 1993). These effects include earlier involvement 
in risky sexual behaviors, higher rates of teen pregnancy, a greater risk of alcohol and other drug 
involvement, and a variety of developmental difficulties (Downs, 1993; Kendall-Tackett, Williams & 
Finkelhor, 1993). Chrysalis students have reported significantly higher prevalence rates of all of 
these behaviors than their peers across Oregon and the nation (Gabriel, Laws, Mitchell & Richen, 
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The powerlessness and stigmatization resulting from abuse often leads many survivors to view the 
world as an unsafe place where they have little control over the events that happen in their lives. 
The developmental delays found among abused individuals may include muted sensory-perceptive 
responses, negative self-image, difficulty in trusting others, or in forming intimate relationships, 
inability to distinguish between thought and action or between feeling and doing, cognitive and motor 
dysfunctions, language disabilities, speech and hearing deficits, and personal and social skill deficits. 
As a result of their histories of personal trauma, these young women are less likely to absorb the 
HIV/AIDS-related education provided in their regular classroom. This evidence provides a further 
rationale for Chrysalis participants benefiting from a HIV/AIDS curriculum tailored to their 
backgrounds, problems and learning styles. 

Overview of Project Activities 

HIV/AIDS Curriculum 

The HIV/AIDS prevention education specialist worked with a variety of public and private community- 
based organizations involved in HIV/AIDS prevention. One key group that contributed its expertise 
to the program was the Cascade AIDS Project (CAP). The HIV/AIDS specialist worked closely with 
CAP to develop a comprehensive and informative educational experience for Chrysalis participants. 
They not only drew upon their own expertise, but used components from existing HIV resources and 
curricula (e.g. Reducing the Risk: Building Skills to Prevent Pregnancy, AIDS: The Preventable 
Epidemic Curriculum, Be Proud Be Responsible, and Power Moves). 

During the school year, the Chrysalis Program group received four separate sessions devoted to 
HIV/AIDS education. The sessions were designed to provide a number of concrete, interactive 
activities and lessons on the following topics: 

• Classroom climate setting; 

• What is HIV, and what are HIV high-risk behaviors; 

• HIV Testing; 

• Abstinence, sex and protection from pregnancy, HIV, and sexually transmitted diseases; 

• Refusal and negotiating skills; 

• Delaying tactics, avoiding high-risk situations; and 

• Accessing and using informational resources for HIV protection. 
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Most of these were conducted by student trainers from the Cascade Aids Project who delivered a 
HIV/AIDS education program called the "Teen to Teen" Workshop. The workshop was led entirely 
by the two student volunteer trainers who presented the information using a variety of interactive 
activities that covered the above mentioned topics. A unique aspect of this workshop was in having 
the delivery and responsibility for the program given to teenagers. The trainers made the information 
interesting to the participants and gave them many opportunities to ask questions and clarify 
information covered. This type of presentation is thought to be effective because teens are more 
comfortable discussing sensitive issues surrounding sexuality with their peers. 

Another component of the presentations was having speakers who were HIV positive talk to 
participants. A powerful component of the program, these speakers spoke about their experience 
of getting the HIV virus, and how the virus has dramatically changed their lives. In addition to having 
the HIV virus, the speakers were victims of childhood sexual abuse. Their stories gave a personal 
focus to a virus that doesn't seem a real threat to many young people because they don't think it can 
happen to them. As one of the speakers poignantly stated, "HIV has really taken over my life." 

The program was delivered to Chrysalis Research group students somewhat differently. Instead of 
four separate sessions, they received the HIV/AIDS curriculum in a single, all day retreat format. 
Each retreat accommodated students from two different high schools and was conducted at a site 
away from their schools. The Research group received the same comprehensive educational 
program as the Program group. To prograrri staff, presenting all the information at once seemed to 
be an effective method of instruction as it made a more powerful impact on the participants. 

Training of Chrysalis Staff 

HIV/AIDS is a topic that comes up regularly for Chrysalis students. In order for case managers to 
be prepared to answer students' questions and concerns, the HIV/AIDS prevention education 
specialist met periodically with each of the case managers to help them understand the issues 
surrounding HIV/AIDS. This kind of assistance gave the case managers information and guidance 
that helped them better serve the Chrysalis population. 

Two case managers had the opportunity to attend several HIV/AIDS training sessions offered during 
the school year. The training sessions, sponsoredrjointly by the Portland School District and the 
Oregon Department of Education, provided instruction in HIV/AIDS prevention curricula. These 
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intensive workshops covered all aspects of HIV/AIDS prevention education, specifically focusing on 
skill building lessons that participants could use in classroom and/or other educational settings. 

All the case managers attended the student HIV/AIDS education program and learned important 
information and skills about HIV/AIDS prevention. They were able to integrate this information into 
their support group sessions with Chrysalis students. 

Evaluation Activities 

Surveys, pre & post 

The primary tool for assessing increased knowledge and preventative attitudes about HIV/AIDS 
among Chrysalis participants was a survey developed by the HIV/AIDS education specialist and the 
evaluation staff. The 17 item instrument was adapted from survey instruments developed by Project 
Action, Center for Health Research and HIV Positive Speakers in the Schools, Portland State 
University. The first seven questions of the survey deal with how the HIV/AIDS virus is transmitted 
(knovriedge); the last 10 questions ask about the change in intentions of behavior to protect against 
HIV/AIDS (attitudes). 

t , 

A pretest was administered to the Chrysalis Research group prior to their attending the one day 
HIV/AIDS retreat workshops. Of the 181 Research girls, 81 (45%) participated in the all day retreats 
and took part in the survey. There was insufficient time to administer a pretest to the Program group 
as their HIV/AIDS training had taken place in separate sessions throughout the school year and the 
survey instrument was not prior to the first session. 

The Chrysalis Program group of 189 students were offered four, one hour sessions of HIV/AIDS 
instruction throughout the school year. Of the 189 girls, two-thirds attended at least one session, 
and about 40 percent of the girls attending three or four sessions. One third of the girls (n=64) 
attended none of the sessions. The survey was mailed to both Research and Program groups in 
June of 1996. Of the 270 surveys mailed, 160 surveys were returned (59%). The returned surveys 
represented 79 Program participants and 83 Research participants. 

Differences in the administration of the survey and in the delivery of the HIV/AIDS curriculum are 
reflected to some extent in the results of the surveys. Those differences include: 
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• The Program group received four separate sessions of the HIV/AIDS prevention curriculum 
throughout the school year in their high school setting, v^hereas the Research group received 
the entire HIV/AIDS curriculum in one-day retreat workshops. 

• The Research group was administered a pre-and post test, while the Program group was given 
just the post test. 

• More Research girls (45 percent) received the entire curriculum compared to girls in the Program 
group. Of the Program group, about one in five girls (21 percent) attended all four sessions. 

• Sixty-two percent of the Research girls (N=50) completed both a pre- and post test. 

Results and Discussion 

The HIV/AIDS education supplement was added to the existing Chrysalis program in order to test 
the hypothesis that delivering a HIV/AIDS education curriculum within an existing prevention program 
would be more effective for participants than providing a HIV/AIDS curriculum that was outside of 
a prevention program. To test this hypothesis, the Chrysalis Research group was used as the 
nonprevention treated comparison group. 

The initial implementation design for the HIV/AIDS curriculum was for both Program and Research 
participants to have the curriculum presented in four separate sessions during the school year. The 
Chrysalis Program students had prescheduled Open Sessions which made it convenient to insert 
the curriculum into those time slots. Scheduling four sessions for the Research group was more of 
a challenge as they had no regular meeting times. Program staff decided that it would be more 
efficient to use an all day retreat format for the Research girls to resolve the scheduling challenge. 

The retreat format turned out to be the most effective way of delivering the HIV/AIDS curriculum. 
The HIV/AIDS curriculum could be presented all at one time and thus the girls who attended did not 
miss any of the important components of the program. 

With this change in implementation and the positive results that change brought to the Research girls 
in the form of more efficient and effective delivery of the curriculum, assessing the differential 
effectiveness of a HlV/AlDS curriculum delivered within an existing prevention program compared 
to a nonprevention treated comparison group could no longer be done. 

The first seven survey items were statements designed to measure knowledge about HIV/AIDS 
contagion and prevention. In Table 1 , the percent of students giving the most knowledge-appropriate 
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response (either "strongly agree" or "strongly disagree,"' depending upon the item) is given for each 
item. The percentage of Chrysalis Program students receiving from two to four curriculum sessions 
(i.e. sufficient dosage) is also displayed. When the data are examined by dosage level, the Program 
girls who attended two or more HIV sessions had improved outcomes and gave a higher percentage 
of appropriate responses were given in five of the seven questions when compared to the Research / 
girls. If the results of the survey responses are compared between all Chrysalis students and 
Research students, then a higher percentage of most appropriate responses were found in the 
Program girls for only three of the seven questions — about willingness to hug or kiss a friend with 
AIDS, and disagreement with the statement, "People who use condoms during sex don't trust the 
person they're with." Thus, dosage appears to be an important factor in showing a positive impact 
on the girls' knowledge regarding HIV/AIDS and its prevention. For both groups, there seemed to 
be greatest understanding that it's okay to hug a close friend who has AIDS, that teens engaging in 
sex without using a condom are at risk of getting HIV, and using condoms during sex does not imply 
a lack of trust in the person you are with. 

Table 2 shows the remaining ten survey items which measure "protective behaviors against 
HIV/AIDS. Again, when dosage level was examined. Program students who participated in two or 
more HIV/AIDS sessions showed a higher percentage of appropriate responses to four of the ten 
survey questions than the Research girls. Four additional questions showed percentages about 
equal to the Research girls, and only two questions showed lower percentages. Whereas, when not 
looking at the dosage level percentages, research girls were more likely to give the most appropriate 
response-doing so for seven of these items. 

The only statistically significant difference between the two groups was found on a key item which 
asked for an estimate of how many friends use condoms when they have sex. After excluding girls 
who responded "Don't Know," only 39.39% of Program girls (40.38% with sufficient dosage) 
responded with "All or most of my friends" compared to 58.73% of Research girls. 

The Program and Research comparisons featured in Tables 1 and 2 were based on post test scores 
only, assessing students' knowledge and attitudes after they had received the HIV/AIDS training. 
Within the Research group, we had the opportunity to document the change in the outcomes from 
the pretraining survey. 

A comparison of pretest and posttest responses for the Research girls shows substantial increases 
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in appropriate responses following exposure to the curriculum. An item that asked whether or not 
the respondent could apply a condom even in the "heat of passion" showed the only significant 
difference with 33.75% of pretest respondents indicating they could compared to 46.15% of posttest 
respondents. Other large increases in appropriate responses were for items regarding fear of kissing 
someone with AIDS, amount of risk when having sex Without a condom, amount of protection 
provided by careful choice of partners, and amount of autonomy felt when dealing with sexual 
partners. For only two items did the percentage of appropriate responses decrease between pretest 
and posttest, and these decreases were minor, and not statistically significant. 



Table 1 



Percentage Making the Most Appropriate Response 
to Knowiedge Questions on Posttest 



Survey item 


All Chrysalis 
Students 


Chrysalis 
Students With 
Sufficient Dosage* 


Research 
Students 
Received Entire 
Curriculum in 


1. 


I would feel comfortable hugging 
a close friend who has AIDS. 


Strongly Agree 
74.03% 


Strongly Agree 
79.87% 


Strongly Agree 
66.27% 


2. 


If I thought my friend had AIDS, I 
would be afraid to give that friend 


Strongly Disagree 


Strongly Disagree 


Strongly Disagree 




a kiss. 


40.79% 


43.37% 


37.35% 


3. 


It’s okay for teenagers to have 
sex without a condom if they 


Strongly Disagree 


Strongly Disagree 


Strongly Disagree 




know each other well. 


53.25% 


56.67% 


65.06% 


4. 


Teenagers are at risk of getting 
infected with HIV if they engage 


Strongly Agree 


Strongly Agree 


Strongly Agree 




in sex without a condom. 


75.33% 


74.71% 


83.13% 


5. 


It’s okay to have sex without a 
condom because your chance of 


Strongly Disagree 


Strongly Disagree 


Strongly Disagree 




getting infected with HIV is very 
low. 


80.52% 


85.67% 


85.54% 


6. 


If teenagers are careful about 
choosing sexual partners, they . 


Strongly Disagree 


Strongly Disagree 


Strongly Disagree 




won’t get infected with HIV. 


48.05% 


61.89% 


54.22% 


7. 


People who use condoms during 
sex don’t trust the person they’re 


Strongly Disagree 


Strongly Disagree 


Strongly Disagree 




with. 


68.83% 


81.95% 


67.47% 



Dosage = Attended 2-4 curriculum sessions 
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Table 2 

Percentage Making the Most Appropriate Responses 
to Behavior Questions on Posttest 





Chrysalis 


Research 


Survey Item 


All Chrysalis 
Students** 


Chrysalis 
Students** 
with Sufficient 
Dosaae*** 


Received Entire 
Curriculum in 
Retreat Format 


8. In general, when you have sex, how 


Every time 


Every time 


Every time 


often do you use a condom? 


or Almost every 


or Almost every 


or Almost every 


time 


time 


time 




52.17% 


56.11% 


51.72% 


9. If you wanted to use a condom every 


Extremely sure 1 


Extremely sure 1 


Extremely sure 1 


time you have sex, how sure are you 


could 


could 


could 


that you could? 


77.05% 


84.03% 


80.60% 


1 0. In the next three months, how likely 


Extremely sure 1 


Extremely sure 1 


Extremely sure 1 


do you think it is that you will start 


will 


will 


will 


using a condom every time you have 








sex? 


51.92% 


61.31% • 


45.76% 


1 1 . How many of your friends do you 


All or most of my 


All or most of my 


All or most of my 


think use condoms when they have 


friends 


friends 


friends 


sex? * 










39.39% 


40.38% 


58.73% 


12. I can put a condom on my 


Very sure 


Very sure 


Very sure 


partner/myself, even during the heat 








of passion. 


60.78% 


59.05% 


60.00% 


1 3. 1 can talk to my partner about using a 


Very sure 


Very sure 


Very sure 


condom, even if 1 don’t know their 








feelings about condoms. 


76.36% 


77.14% 


80% 


1 4. 1 can refuse to have sex with my new 


Very sure 


Very sure 


Very sure 


partner if they won't use a condom. 








even if 1 really like them. 


75.44% 


76.36% 


79.03% 


1 5. What would you say are your 


Very unlikely 


Very unlikely 


Very unlikely 


chances of getting HIV virus during 








the next 1 2 months? 


75.68% 


82.84% 


84.15% 


1 6. How much do you feel you can do to 


A lot 


A lot 


A lot 


keep from getting the HIV virus in the 








next year? 


86.84% 


93.12% 


89.16% 


17. Have you done anything in the past 


Yes 


Yes 


Yes 


month to help decrease your 








chances of qettinq HIV? 


72.00% 


75.66% 


81.93% 



“Excludes program girls who don’t have sex 
“‘Dosage = Attended 2-4 curriculum sessions 
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When comparing Chrysalis Program girls to Research girls, survey responses showed that 
knowledge about HIV/AIDS contagion and prevention and intention to change behavior to protect 
against HIV infection was equal to and often times higher for Program girls who had attended 
between two to four curriculum sessions than Research girls who had received the entire curriculum 
at one time (see Figure 3 below). 

Overall, both the intervention and control groups benefited from the program, but the control group 
received the benefit of more effective implementation of the curriculum. In retrospect, the curriculum 
needed to be delivered the same way to both groups to yield a valid comparison of effectiveness. 



Figure 3 

Program Girls Only 

(low dosage = 0 or 1 session; high dosage = 2, 3, or 4 sessions) 
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APPENDIX D 

Evaluation of Supplementary Grant: 

■ Media Literacy Project 
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Chrysalis MEDIA: Media Education Demonstration In Action Project 

Purpose of the Project 

Chrysalis case managers report that a wide range of problems related to poor body image, low self- 
esteem, the lack of a strong sense of individual identity, and deep concern about how they are 
suppose to act in relation to both male and female peers are pervasive among the young women 
participating in Chrysalis. For example, eating disorders are common and these can relate to other 
health risk behaviors. Many who smoke cigarettes are reluctant to quit because they believe they 
will gain weight. Chrysalis project evaluation results indicate that more than 60 percent of these 
young women are trying to lose weight, some by unhealthy methods (diet pills, vomiting). 

The issues observed by the case managers are consistent with the research literature, which cites 
a broad range of harmful effects resulting from childhood sexual abuse and other forms of 
maltreatment, including greater risk of ATOD involvement, school failure, earlier involvement in risky 
sexual behaviors, teen pregnancy, and a variety of developmental difficulties (Finkelhor and Browne, 
1987). The powerlessness and stigma associated with abuse tend to drain the world of sense and 
meaning. As a result, these students often become "passive actors, who do not think they can help 
•themselves, make choices, or take action that will make a difference" (Boyer and Fine, 1992). 
Studies in the prevention field of risk and resiliency factors among high-risk youth have emphasized 
the importance of opportunities to exercise personal responsibility and for meaningful involvement 
in their schools and communities (Benard, 1991; Wehlage, 1989). The Chrysalis MEDIA Project was 
designed to empower students to think critically about the information they receive in a multimedia 
society and the decisions they make daily based on influences such as advertising and other media. 
The involvement of Chrysalis students in actually using the media to educate their peers and the 
wider public was expected to be a highly empowering experience. The chance to work directly with 
professional women who are working as film makers, educators, and actresses was also designed 
to encourage participants to consider various careers and the skills required to succeed in them. 

The evidence is clear that the media "message environment" — encompassing television, movies, 
the news, magazines targeted to specific populations such as female teenagers, advertising — has 
powerful, subtle, and often negative impacts on feelings, attitudes, and behaviors which are 
associated with ATOD use and other correlated risk factors (e.g. Wallack, 1985; Resnick, 1990). 
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The literature on effective mass media campaigns and communications approaches also cites the 
importance of customizing presentations for various subgroups such as adolescent females (e.g., 
Pierce, 1993; Maher, 1992; & Bandy et al., 1983). Effective prevention campaigns need to present 
factual information about the lifestyle, physiological, emotional, and legal consequences of ATOD 
use (Mayton et al., 1990, Flay et al., 1989). 

\ 

Three of the Chrysalis Open Sessions \were devoted to teaching the skills needed to identify and 
critique the messages various media disseminate regarding women (including adolescents and 
women from various cultural and socioeconomic groups), and the impact of these messages on self- 
image, use of ATOD and other unhealthy behaviors. 

Overview of Project Activities 

The MEDIA project consisted of three major activities or components: training sessions for all 
Chrysalis Program students during the school year; a production workshop for a select group of ten 
students during the summer; and a proposed training retreat for ten Chrysalis students to develop 
an educational program for middle school students. 

Media Literacy Training 

Three, one-hour open sessions were offered during the school year for all Chrysalis Program 
participants by a media specialist hired for the year. The sessions, held at the twelve participating 
high schools, examined the following areas: 

• Gender roles and stereotypes (including attitudes toward various cultural groups and specific 
female populations — disabled, elderly, or poor women, for example) found on television and 
in other media targeted to female adolescents; 

• How to "read" an advertisement and other media in terms of the underlying message being 
disseminated; and 

• Fact vs fiction in the media — encouraging students to become analytical and critical thinkers. 

Participants were encouraged to keep journals recording their observations of the content and form 
of various media messages. This activity was used to spur discussions and counter the tendency 
to be unquestioning consumers of media such as television and movies, whose rapid-fire images 
prevent active interaction or questioning by viewers of what they see and has on the screen. 
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The media literacy activities provided during the regular school year also benefited the Chrysalis 
case managers and one or two high school teachers responded with an interest in learning how to 
integrate the subject matter into their classroom curricula effectively. 

Media Production Workshop 

Ten Chrysalis youth were recruited to apply what they had learned about media literacy during the 
school year to designing and producing a 30-second animated public service announcement (PSA) 
and a three-minute videotape. The diverse group of students attended a two-week media literacy 
workshop in June 1996. The workshop was led by two professional female animators/ filmmakers 
from the Northwest Film Center's Video/Film Maker-in-Schools Program and by other Chrysalis staff. 

The use of animation allowed participants to avoid confidentiality issues while addressing important 
and potentially highly personal subject matter in a creative and visually arresting manner.. The 
content and form of the two products was determined by the participants who used the animated film 
format to address the issues of media influence, ATOD use, gender and abuse. 

The project was a collaborative work of all the students. The theme and story line of the PSA, "Fight 
Girl Poisoning," was created and developed through consensus by the group. Each person had a 
role to play in the production and was responsible for getting their work done in the time frame 
allotted. Some of the girls drew the dozens of pictures needed for each frame of the PSA. Other 
girls were responsible for developing the script and the audio portions of the video. 

The three-minute video, "Girl Power," required each girl to develop a small mini-story about an ATOD 
issue that was important to her, draw the pictures and provide the script and audio for their piece. 
Each day was full of acquiring new knowledge and skills. The participants learned what animation 
was, what it entailed and how to apply that knowledge to the development of their own animation 
work. They were introduced to a variety of film equipment and how to use it. 

An important aspect of the workshop was the commitment that each participant was willing to show 
during the two week course. All the young women were required to attend the workshop every day 
and to arrive on time. This kind of commitment is important as the project needed everyone's input 
and work. Othenwise, the project would not have succeeded as intended. This commitment of their 
time was also significant because many of the girls had attendance problems in the past school year. 
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Intended Follow up 

Pending the acquisition of additional funding resources, a training retreat is planned in which ten 
interested Chrysalis' high school girls will develop a presentation to accompany the "Fight Girl 
Poisoning" PSA and offer this program to middle school health classes. The proposed two and one- 
half day retreat will involve Chrysalis Project staff and a professional actress from the Artists-in- 
Education program operated by Young Audiences of Oregon who will coach the students and staff 
on how to design and deliver an effective education presentation that will reinforce and build on the 
material covered by the three-minute animation videotape. As part of the retreat training the PPS 
Television Services Department will arrange for video-taping equipment to be available, so that the 
students can review, critique, and refine their presentations. 

During the 1996-97 school year, the Chrysalis students who attended the retreat will deliver the 
presentations and show the animated "Fight Girl Poisoning" in the middle school health classes. The 
students will also help arrange other opportunities to air the public service announcement and 
encourage new Chrysalis participants to become involved in educating their middle school peers. 

Evaluation Activities and Results 

The media literacy curriculum and the two-week workshop were designed to increase students’ skills, 
knowledge, and attitudes related to the content and form of media messages and their impact on 
self-image, ATOD use, and health issues. It was anticipated that the media literacy training would 
empower youth to better evaluate the impact of visual/print images on women of various cultures. 

Process and outcome evaluation activities included structured observations and interviews of 
participants, survey administration and review of instructor notes and student evaluations. All of 
these activities were conducted to determine to what extent the MEDIA Project had accomplished, 
the desired effects on the target population. 

Program Observations: Two-week Media Workshop 

Observation of the media workshop was done by evaluation staff during the two-week workshop. 
On the day of the observation, workshop participants were busy creating the pictures and developing 
the script for the PSA. All girls were actively engaged in their tasks and were thoroughly enjoying 
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the experience of working together on such a creative endeavor. The two professional filmmakers 
from the Northwest Film Center moved about the room, assisting girls with the various tasks they 
were doing, answering questions and directing the next steps toward completion of the PSA. 

The "storyboard" (ie. story line) for the PSA the girls created was posted on the workshop walls so 

/ 

the girls and staff could refer to it to keep track of what they were working on and what needed to 
be accomplished next. Some girls were occupied with doing the numerous drawings required for the 
animated PSA. Others were assigned the task of making title cards for the video. Another group 
worked on cutouts that were used for the animation in addition to the drawings. "Twenty-four frames 
per second" seemed to be the words that came out of everyone's mouth at some time during the 
course of the day. It takes twenty-four pictures to make one second of film so the number of 
pictures required for the thirty second PSA was 720. Each of these pictures and title cards had to 
be filmed which added additional time to a tight schedule. On the day of the observation, the script 
and audio portions were just beginning to be developed and would later be recorded by the girls. 

If the girls were not working on the PSA, they were working on their "flip books". The flip books were 
each girls' contribution to the three minute video. They developed a message about an ATOD issue 
of concern to them that they wanted to convey and then drew their ideas and wrote a brief script 
about it. The flip books are a series of pictures that when flipped through, create moving images. 
These flip books were time consuming because it took a lot of work to draw the required pictures to 
create the animation. Each flip book was filmed painstakingly using a variety of sophisticated film 
equipment that the girls were taught to operate. 

There was some tension among participants. This observation was confirmed later in the structured 
interviews with participants and in the review of student evaluations. The tension was created in 
large part by the short time frame the group had to complete the two videos. There was a lot of 
information and skills for instructors to teach which required a lot of attention and commitment by the 
participants. This naturally created stress and tension among and between instructors and students. 
The tension did not impede the progress of the workshop and did not carry over to how the 
participants felt about the workshop experience. 

Program Observations-Video Premiere 

Project Chrysalis held a special showing of the PSA and short video the girls developed in the media 
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literacy animation workshop for participants, friends and family and staff. It was held at the 
Northwest Film Center's auditorium, next to the Portland Art Museum, in downtown Portland. The 
evening program consisted of welcoming workshop participants and their family and friends and then 
showing a slide presentation of the girls hard at work on their project during the two week workshop. 
The PSA, "Fight Girl Poisoning" and the film, "Girl Power" were shown on the big screen which awed 
and impressed not only the workshop participants but other members of the audience as well. The 
pride that the girls and the rest of the audience felt about the workshop results was palpable. Each 
girl was given her own copy of the PSA and film as well as a special award for her individual 
contribution to the workshop. A reception followed the premiere where attendees could enjoy 
refreshments, visit together and discuss and praise the efforts and accomplishments of the media 
workshop participants. 

Instructor's Notes 

Chrysalis staff kept a journal of the daily activities of participants during the workshop. As recorded 
in the notes, a good portion of the first week of the workshop was used to familiarize the girls with 
the process of animation as well as brainstorming, discussing and writing down the messages and 
concepts the girls wanted to convey through the PSA and the three minute film. As mentioned 
earlier in this report, attendance by the girls was important for the success of the project so the staff 
kept track of the girls attendance and reported that nearly all the girls had 100 percent attendance. 

During the course of the workshop. Chrysalis staff would open the workshop with checking-in to see 
if there were any concerns, problems or other information they or the girls needed to discuss. At 
the end of the day, staff would debrief about how the day went , make any announcements that were 
necessary and talk about what needed to get accomplished. 

Participant Interviews 

Two girls who participated in the summer media workshop were interviewed by evaluation staff to 
get a more in depth view of their experience in the workshop (see interview questions at end of this 
report). Both girls enjoyed participating in the workshop. They found the learning experience both 
fun and interesting. The girls felt that the workshop needed to be longer because of the amount of 
work that needed to be done. The lack of time created a great deal of stress and tension among the 
participants and staff which they both said was the most difficult thing to deal with in the workshop. 
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The girls were asked if they had applied what they had learned in the media literacy classes during 
the school year to what they did in the workshop. They said that the media classes gave them the 
background information about how the different types of media portray women. One interviewee 
stated that she could really see the difference in how men and women are portrayed differently by 
the media. She wanted to create a PSA in the workshop that would be taken the same way by both 
males and females, in other words, the message conveyed would be clear to both genders. 

The girls felt proud, for themselves and for the other girls, about the end results of the PSA, "Fight 
Girl Poisoning" and the three minute video, "Girl Power." As one girl said when she saw the final 
results of her work, "That's mine. I did that." 

Asked what the overall message of "Fight Girl Poisoning" was, one girl stated that the PSA shows 
that "we need to stop the negative images that girls get about their minds and bodies — that they 
need to be themselves." The young woman went on to say that "if the commercial [PSA] gets to 
one girl or guy who needs it, that makes all the difference in the world." 

Surveys 

Pre- and post test surveys were administered to participants who attended the media literacy open 
sessions. A total of 108 surveys were completed by participants with pretest surveys numbering 72 
and post test surveys numbering 36. Of the three sessions offered the Program group during the 
school year, 60 percent (n=1 14) of the 189 Chrysalis students attended at least one of the sessions. 
About one in eight (12 percent) of the Program girls attended all three sessions. Since the surveys 
were not linked to a project identification number, these participation differences could not be related 
to any differences in survey responses. 

Discussion and Summary of Major Findings 

The first 13 questions on this survey asked the participants to rate their knowledge pertaining to such 
things as television advertising and public service announcements and to rate their ability to 
recognize distortion of truth, manipulation, and biases found in the media. These results are given 
in Table A. Except for one question, "How is your ability to recognize media content that may be 
biased for or against sexual, ethnic or cultural groups?," a comparison of responses consistently 
shows an increase in the percentage of participants who rate their skills and knowledge as "Very 
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Good" from pretest to posttest.' Statistically significant increases in the percentage of "Very Good" 
responses occurred when participants were asked to rate a) their skills as a critical viewer, b) their 
knowledge of how to produce a video message, and c) their ability to communicate prevention 
messages to the public. The increase in "Very Good" responses was also nearly significant for the 
ability to recognize media marketing to ethnic groups between pretest and posttest. In addition, the 
number of participants who responded "Don't Know" to each question lowered dramatically from 
pretest to posttest, indicating that familiarity with the issues had been greatly enhanced. 



Table A 

Percentage Responding “Very Good” and “Don’t Know” on Pre and Posttests 





Very Good 


Don’t Know 


Item 


Pre 


Post 


Pre 


Post 


1 . How are your skills as a critical viewer of television? 


19.44% 


27.78% 


31 .94% 


25.00% 


2. How are your skills as a critical viewer generally? * 


19.44% 


33.33% 


33.33% 


8.33% 


3. How good is your understanding of media influences 
directed towards teenagers and women? 


34.72% 


40.00% 


18.06% 


5.71% 


4. How is your ability to recognize differences between 
fact and fiction in commercial advertising? 


59.72% 


63.89% 


6.94% 


0.00% 


5. How is your ability to recognize media content that may 
be biased for or against sexual, ethnic or cultural 
groups? 


58.33% 


47.22% 


11.11% 


5.56% 


6. Your knowledge of the purpose of a public service 
announcement. 


36.11% 


47.22% 


18.06% 


8.33% 


7. Your knowledge of how to produce a video message. * 


15.49% 


25.00% 


45.07% 


11.11% 


8. Your ability to judge media facts from fiction. 


30.00% 


38.89% 


11.43% 


5.56% 


9. Your ability to recognize sex-role stereotypes in 
advertising. 


53.52% 


66.67% 


9.86% 


0.00% 


10. Your ability to recognize media marketing to ethnic 
groups. 


30.99% 


55.56% 


19.72% 


5.56% 


1 1 . Your ability to be a critical viewer of television. 


33.80% 


44.44% 


28.17% 


8.33% 


12. Your ability to communicate prevention messages to 
your peers. 


38.03% 


41.67% 


19.72% 


11.11% 


13. Your ability to communicate prevention messages to 
the public. * 


28.17% 


41.67% 


26.76% 


2.78% 
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Table B summarizes the remaining ten statements which measured attitudes and opinions about 
community exposure to drug and alcohol abuse information through the media and the need for this 
type of exposure in the Portland area. The posttest showed a sharp increase in the percentage of 
participants who agreed with statements indicating that Portlanders have access to friendly and 
humane public services and support media campaigns aimed at preventing drug and alcohol abuse 
and showed a moderate increase in support for additional funding for alcohol and drug prevention 
programs in schools. 

Only about one third of pretest participants thought that Portlanders have appropriate media 
exposure to information about alcohol and drugs, and this rate remained the same for the posttest. 
A low percentage (about 18%) agreed with the statement, "Using illegal drugs is okay if you don't 
get addicted" in both pretest and posttest, and a high percentage (slightly over 70%) on both tests 
thought more money should be spent on public awareness of substance abuse issues. 

In most cases, the percentage of participants who indicated they were "Undecided" on the issues 
presented by these questions decreased from pretest to posttest. The increases occurred on two 
statements about the ease of purchasing cigarettes or alcohol by minors in our community and two 
statements about the need for additional money to pay for more youth activity programs and more 
law enforcement. 
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Table B 

Percentage Agreeing, Disagreeing, or Undecided on Pre and Posttests 



Item 


Agree 


Disai 


gree 


Undecided 


Pre 


Post 


Pre 


Post 


Pre 


Post 


14. In general, the citizens of Portiand have 
access to friendiy and humane pubiic 
services. 


41.67% 


62.86% 


11.12% 


17.14% 


47.22% 


20.00% 


15. in generai, the citizens of Portiand support 
media campaigns aimed at reducing or 
preventing aicohoi & drug use. 


49.30% 


73.53% 


19.72% 


1 1 .76% 


30.99% 


14.71% 


16. in generai, the citizens of Portiand have 
appropriate media exposure about aicohoi 
and drugs. 


32.57% 


34.28% 


29.58% 


31.43% 


38.03% 


34.29% 


1 7. It's easy for minors to buy cigarettes in our 
community. 


94.45% 


82.86% 


2.78% 


2.86% 


2.78% 


14.29% 


1 8. It's easy for minors to buy alcohol in our 
community. 


72.22% 


61.76% 


18.06% 


8.82% 


9.72% 


29.41% 


1 9. Using illegal drugs is okay if you don't get 
addicted. 


18.31% 


18.18% 


54.93% 


57.57% 


26.76% 


24.24% 


20a. Additional money should be available for 
> more public awareness of alcohol and 

drug problems. 


70.84% 


73.53% 


8.33% 


5.88% 


20.83% 


20.59% 


20b. Additional money should be available for 
more community youth activity programs. 


81.95% 


82.36% 


4.17% 


2.94% 


13.89% 


14.71% 


20c. Additional money should be available for 
more law enforcement. 


53.52% 


54.54% 


28.17% 


21.21% 


18.31% 


24.24% 


20d. Additional money should be available for 
more alcohol/drug prevention programs in 
schools. 


69.44% 


78.78% 


15.27% 


6.06% 


15.28% 


15.28% 
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Media Literacy Animation Summer Workshop 
June 17*28, 1996 
interview Questions 



What was the most difficult or demanding part of participating in this workshop? 



What were some of the useful things you learned from participating in this workshop? (i.e. knowledge 
and/or skills about animation film making; knowledge and/or skills about media literacy.) 



Do you think that these skills, knowledge, etc. will be useful to you in the future? If so, how? (Give 
examples). 



The thing I liked best about the workshop experience was 



The thing I liked least about the workshop experience was 



Did you apply what you learned in the media literacy classes you attended during the past school 
year to what you did in this workshop? If so, in what way? 



How well did this workshop match your expectations of it? In other words, did it turn out to be what 
you thought it was going to be or was it quite different? 



What was the overall message of the PSA you contributed to? Do you think that message came 
across in the PSA? 



How did you feel when you saw the end results of your PSA on the "big" screen? (Were you proud, 
disappointed, satisfied, etc.?) 



Did you feel that your work on the PSA was worthwhile? In what ways? 



What did you think about the end results of the film. Girl Power that was created from your flip 
books? 
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APPENDIX E 

Evaluation Instruments, Year 2 




■ 


Intake Form 




■ 


Needs Assessment 




■ 


Case Manager Contact Log 




■ 


Chrysalis Activity Log 




■ 


Risk and Resilience Index 




■ 


Case Study Interview Protocols 




■ 


Girls Empowerment Questionnaire 




■ 


(pre and post) 

Challenge Course Questionnaire 
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Student 



Date 



Student I.D.# 

Age Grade 

Student Phone 

Case Manager _ 



School/Program 



Date of Birth 



Ethnic Group . 



Student Address . 
Phone 



. Sexual Orientation: □ Heterosexual □ Lesbian □ Bi-sexLial- 



All participants must have experienced some kind of abuse. 



Type of Abuse; (check any) 
Was Abuse Reported? 

Date Abuse was Reported: . 
Current Status: (check one) 



□ Physical □ Sexual □ Psychological 

OYes QNo □ Don’t know 

Location (City/State) 

□ Inactive □ Active 

CSD Caseworker Phone 



1-5 



(^) here if none of the above categories appiy. 

Students must have two (2) additional risk factors from the list below to be eligible to participate in this project. 
Check all that apply: 



6 


□ 


Child of substance abuses 


□ 


Parent with severe emotional problems 


18 


7 


□ 


Uses alcohol/drugs or past history of AOD use 


□ 


Learning disabilities/special education 


16 


6 


□ 


Smokes cigarettes/uses tobacco 


□ 


Physically disabled or in chronic pain 


20 


9 


□ 


Violent or other delinquent behavior 


□ 


Pregnant 


21 


10 


□ 


Past/present involvement in juvenile justice system 


□ 


Teen parent # and aender of children: 


22 


11 


□ 


Gang member or involved with gangs 


□ 


At risk for HIV or other STDs 


23 


12 


□ 


Witness to violence 


□ 


Runaway/out of home placement 


24 


13 


□ 


Victim of domestic violence 


□ 


School dropout/chronic school failure 


25 


14 


□ 


Victim of school violence 


□ 


Economically disadvantaged/homeless 


26 


15 


□ 


Know someone who was murdered 


□ 


Parent Involved In criminal justice system 


27 


16 


□ 


Suicide tendencies (current thoughts or past 


□ 


Parent incarcerated 


28 






attempt) 


□ 


Other 


20 



17 □ Severe emotional problems 



□ Chrysalis Case Manager 


□ School Counselor 


Referral Source: 

□ Teen Parent Program 


□ Parent 


□ 


Self 


□ Middle School Counselor 


□ Juvenile Justice 


□ School Nurse/Clinic 


□ Special Education 


□ 


Teacher 


□ Other Student 


□ Alternative School 


Name/Phone: 









If student is 18 years or older and signs own consent, answer these questions; 

Can information about program be sent to parents? □ yes □ no 

Can information about program be sent to student by mailing to home address? □ yes □ no 
Student initials here. 




Project Chrysalis is a federally funded grant. For more information, call Stevie Newcomer, Portland Public Schools, Student Services, 280*5840 ext. 283. 
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C H R Y 



A L I 






Needs Assessment 



liiPI 



Ail 



Part I 



S student to complete) 






JPS'^vMIS ;■ -Rif 



Here are a number of ideas or activities that your Chrysalis Case Manager could assist you 
with this year. Look through the list and, pick three that interests you the most. 

“My life would be better if I got some help with “ 



•". .. . .=-:"v ' 

V A V>' 



□ 



School work (tutors, books, supplies)! 






D Transportation (driver’s training, business, bus passes/tickets) 

D Medical and/or dental care (referrals) 

D Participating in school/ after-school activities (fees, equipment, clothing) 
D Getting a job (job placement services, work clothes) 

D Hobbies (supplies, materials, fees for classes) 

D Money to buy some things for myself (clothing, makeup, etc.) 

D Other things like: 



When you finish, your Case Manager will have a few follow-up questions for you. 



12/7/95 
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Chrysalis Needs Assessment 
Part II 



(Case Manager Recording Form) 



Once your student has identified the three areas she is most interested in receiving 
assistance, ask her the following two questions about each area: 

1 . What is keeping you from being where you want to be with ? 

2. What can we do to help you get there? 

Record answers to the above questions for each area below. 

Area #1 : 



Area #2: 



Area #3: 
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Directions for Compieting Chrysaiis Activity Log: 

Please complete the Chrysalis Attendance / Activity Log using the following topic numbers 


and descriptions of group sessions. Case managers should confirm the date and topic 


of each support group with the Waverly co-facilitator at their school. 


TOPiC 


Recommended 


Support Group Sessions 


Number of Sessions 


1 Who are we? What is a Chrysalis? 


1-2 


2 What is abuse? 


1 


3 Storytelling 


1 


4 Secrets/Trust 


1-2 


5 Storytelling through the family 


2 


6 Trauma and its effects 


2 


7 Shame/blame/depression 


1 


8 Boundaries 


1 


9 Powerlessness 


1 


1 0 How we see ourselves 


1-2 


11 Intimacy 


1-2 


12 Sex and relationships 


1-5 


13 A celebration of women 


1 


Open Sessions and Other Activities 


Recommended 
Number of Sessions 


14 Challenge Course 


1 


15 Girls Empowerment 


1 


16 Year End Celebration 


1 


1 7 Media Literacy 


3 


18 HIV/AIDS Education 


4 


19 Alcohol/Drug Prevention 


1 


20 Tobacco Awareness 


1 


21 Eating Disorders/Nutrition 


1 


22 Female Role Models 


1 


23 Trust/Team Building 


1 


24 Student Portfolios/Journalling 


1 



er|c 
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Case Manager: School: Log for the Month of: 




00 

CP 



o 

« 



CO 



1 5 > 
Q CP 
.0 
<6 
3 

73 



o 

3 

£ 




Portland Public Schools 
Proj ect Chrysalis 

Risk & Resiliency Index 




Developed by: 

EMT Associates 
Sacramento, California 



Adapted for use by: 

RMC Research Corporation 
Portland Oregon 



Spring 1996 



PPS Chrysalis: Risk & Resiliency Index 



The following questions will take approximately 30 minutes to complete. In answering them, we 
are asking two things of you. 

First: We need you to read carefully some questions about you and your family. 

It is important that you answer eyery question. 

Second: It is very important that you answer each question truthfully. Although we 

haye asked you to giye us your name, your answers will be kept 
completely confidential, and used to help us be more helpful to you. 



Part I 

You are going to read a lot of sentences. For each of these sentences, please read along and mark 
the circle below the answer that is closest to how you feel today about what the sentence says. 

Mark in YES! If you belieye yery strongly that the sentence is true for you, that it 

is the way you feel almost all of the time. 

Mark in yes If you sort of agree that the sentence is true for you, that it is the 

way you feel most of the time. 

Mark in no If you sort of belieye the sentence is false for you, that you do not 

feel that way most of the time. 

Mark in NO! If you belieye yery strongly that the sentence is false, that you almost 
neyer feel this way. 



Let's practice by reading the following sentence: 

I like pepperoni pizza. YES! yes no NO! 

If you really like pepperoni pizza, it is one of you fayorite foods, you would mark YES!, if you really 
don't like it, you can't stand to eat it, you would mark NO!. If you sort of like it, you would mark 
yes, if you sort of don't like it, you would mark no. 

Okay. We are ready to start. 
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Parti 







YES! 


yes 


no 


NO! 


1. 


I can tell my parents the way I feel about things 


.... O 


O 


O 


o 


2. 


I will probably die before I am thirty 


.... O 


O 


O 


o 


3. 


One of my problems is I cannot get down to work when I should 


... o 


O 


O 


o 


4. 


There are people I can depend on to help me if I really need it 


.... o 


O 


O 


o 


5. 


I get along well with other people 


.... o 


O 


O 


o 


6. 


If I can’t do a job the first time, I keep trying until I can 


.... o 


O 


O 


o 


7. 


I can be trusted 


.... o 


O 


O 


o 


8. 


I am afiraid my life will be unhappy 


.... o 


O 


O 


o 


9. 


There is an adult I could talk to about important decisions 


.... o 


O 


o 


o 


10. 


School is a waste of time 


.... o 


O 


o 


o 






YES! 


yes 


no 


NO! 


11. 


I give up on things before completing them 


.... o 


O 


o 


o 


12. 


Bad things happen to people like me 


.... o 


O 


o 


o 


13. 


Helping others makes me feel good 


.... o 


o 


o 


o 


14. 


I like to help other people who have helped me 


.... o 


o 


o 


o 


15. 


My life is all mixed up 


.... o 


o 






16. 


It is hard for me to make firiends 


.... o 


o 


o 


o 


17. 


When I have something unpleasant to do, I stick to it until I finish it . . 


.... o 


o 


o 


o 


18. 


There is a trustworthy adult I could turn to for advice if 












I were having problems 


.... o 


o 


o 


o 


19. 


I try hard to do well in school 


.... o 


o 


o 


o 


20. 


I like to do things with my family 


.... o 


o 


o 


o 


21. 


I can do most things I try 


.... o 


o 


o 


o 
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YES! 


yes 


no 


NO! 


22. 


When I decide to do something, I go right to work on it 


.... O 


O 


O 


O 


23. 


If I study hard, I will get better grades 


.... O 


O 


O 


O 


24. 


When I am mad, I yell at people 


.... O 


O 


O 


O 


25. 


If I don't understand something, I will ask for an explanation 


.... O 


O 


O 


o 


26. 


My friends respect me 




O 


O 


o 


27. 


I always like to do my part 


.... O 


O 


O 


o 


28. 


People usually like me 




O 






29. 


Sometimes I break things on purpose 


.... O 


O 


O 


o 


30. 


I am often too embarrassed to ask questions 


.... O 


O 


O 


o 


31. 


When I try to learn something new, I soon give up if 












I don't succeed right away 


.... O 


O 


O 


o 


32. 


There is no one I can depend on for help when I really need it 


.... O 


O 


O 


o 






YES! 


yes 


no 


NO! 


33. 


It is important to do your part in helping at home 


.... O 


O 


O 


o 


34. 


I like the way I act 


.... o 


O 






35. 


I know when I should take responsibility and when I shouldn't 


.... o 


o 


O 


o 


36. 


If you work hard, you will get what you want 


.... o 


o 


O 


o 


37. 


There is no adult I feel comfortable talking about my problems with . . 


.... o 


o 


,o 


o 


38. 


I would like to quit school as soon as I can 


.... o 


o 


o 


o 


39. 


There are people I can count on in an emergency 


.... o 


o 


o 


o 


40. 


I enjoy talking with my family 


.... o 


o 


o 


o 


41. 


Helping others is very satisfying 


.... o 


o 


o 


o 


42. 


I like the way I look 


.... o 


o 


o 


o 


43. 


If I feel like it, I hit people 


.... o 


o 


o 


o 


44. 


I often disappoint people 


.... o 


o 


o 


o 


45. 


I am responsible for what happens to me 


.... o 


o 


o 


o 



o 

ERIC 
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Part II 



The answers for these questions are different than those we just finished, so we have to read them very carefully. 
Remember that it is very important that you answer each question. Okay, we are ready to start! 



1. Where you are living now, what adults live with you (shade in all that are true)? 



o 


Your mother 


o 


Your father 


o 


Your stepmother 


o 


Your stepfather 


o 


Other adult relatives such as 


o 


Other adults 




grandmother or aunt 


o 


I live on my own. 



2. How many brothers/stepbrothers and 
sisters/stepsisters do you have? 



3. How many children, excluding yourself, live with 
you at home? 



4. Here is a list of things that are true in some families and not in others. Please answer "yes" if 
each statement is usually true of your family, and "no" if it is not. 



The rules in our house are clear 

1 have a clear time when 1 have to be home 

1 have a regular time and place to do homework 

My parents often do not know where 1 am 

When I do something wrong, 1 know what my parents will do 
1 have regular chores to do at home 



yes 

O 

O 

O 

O 

O 

O 



no 

O 

O 

O 

O 

O 

O 



5. In the last year have you done the following? 

Cjo to church or Sunday school or religious services regularly . . . 

Play on organized sports teams for kids 

Belong to any other clubs like Girl Scouts, YWCA, Campfire Girls 
Belong to a gang 



y^^es 

. O 
. O 

. o 
. o 



no 

O 

O 

O 

O 
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6. The following list indicates things that might happen to young people, or things that young 
people might do. Please indicate whether you have done these things three or more times in 
the last month, only once or twice or not at all. 





3 or more 


Once or 


Not at 




times 


twice 


aU 


CJot sent to the principal's office or had detention 


O 


O 


O 


Skipped school for a whole day 


O 


O 


O 


Purposely damaged other people's property 


O 


O 


O 


Stole something 


O 


O 


O 


Got into a fist fight 


O 


O 


O 


Got stopped by the police 


O 


O 


O 



7. Have you ever wished that either one or both of your parents would drink less? 

O Yes O No O No, my parents do not drink 

8. Here are some other things that happen in some families and do not happen in others. Please 
indicate whether these things happen in your family. If you don't live with your parents, think of 
the adult(s) who you do live with when we ask about parents. 

• all the time (that is, every day or alm ost every day) 

• often (once a week or so) 

• not very often (less than once a week) 

• never 



My whole family eats dinner together 

Parents or other adults help me with my 
homework 

I go to a movie or out to dinner with my 
parents 

My family members argue 

I talk to my parents about school 



AU t ;e 


Often 


Not very 


Never 


time ' 




often 




O 


O 


O 


O 


O 


O 


O 


O 


o 


O 


O 


O 


o 


O 


O 


O 


o 


O 


O 


O 
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This IS the third and Gnal portion of the question- 
naire, We will be done soon. Remember, your 
answers will be kept completely confidential. No 
one will know bow you, as an individual, 
answered these questions. It is very important 
that you answer honestly. 

Please circle the answer that best describes you 
for each of the remaining questions on the survey. 

The next four questions ask about body weight, 

1. How do you think of yourself? 

a. Very underweight 

b. Slightly underweight 

c. About the right weight 

d. Slightly overweight 

e. Very overweight 

2. Which of the following are you trying to do? 

a. Lx>se weight 

b. Gain weight 

c. Stay the same 

d. I am not trying to do anything about my 
weight 

3. During the past 7 days, which one of the following 
did you do to lose weight or to keep from gaining 
weight? 

a. I did not try to lose weight or keep from 
gaining weight 

b. I dieted 

c. I exercised 

d. I exercised and dieted 

e. I used some other method, but I did not 
" exercise or diet 



4. During the past 7 days, which one of the following 
did you do to lose weight? 

a. I did not try to lose weight or keep from 
gaining weight 

b. I made myself vomit 

c. I took diet pills 

d. I made myself vomit and took diet pills 

e. I used some other method, but I did not 
vomit or take diet pills 

The next six questions are about your attitudes and 
your friends^ attitudes toward cigarettes, alcohol, 
and other drugs, 

5. If you use alcohol or other drugs, will you have 
more health problems than other people? 

a. Yes 

b. Probably 

c. I don’t think so 

d. No 

6. If I don’t use alcohol or other drugs I will be 
happier. 

a. Strongly agree 

b. Agree 

c. Disagree 

d. Strongly disagree 

7. What would your best friends think if you got 
drunk once in a while? 

a. They would be angry with me 

b. They would be a little upset 

c. They wouldn’t care one way or the other 

d. They would accept me 

e. They would be glad 
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8. Smoking cigarettes fits with the kind of life I want 
to lead. 

a. Yes, definitely 

b. Yes, probably 

c. Probably not 

d. Definitely not 

9. How do you think your closest friends feel about 
this statement: 

People who use drugs are stupid 

a. Strongly agree 

b. Agree 

c. Disagree 

d. Strongly disagree 

10. Getting drunk every now and then fits with the 
kind of life I want to lead. 

a. Yes, definitely 

b. . Yes, probably 

c. Probably not 

d. Definitely not 

The next few questions are about CIGARETTES. 

11. On how many DAYS did you smoke a cigarette in 
the LAST MONTH (30 days)? 

a. None 

b. 1 or 2 days in the last month 

c. 3 to 5 days in the last month 

d. 6 to 9 days in the last month 

e. 10 to 19 days in the last month 

f. 20 to 31 days in the last month 



12. On the days you smoke cigarettes, how many do 
you smoke? 

a. Less than 1 cigarette 

b. 1 or 2 cigarettes 

c. 3 to 7 cigarettes 

d. About half a pack of cigarettes 

e. A pack or more of cigarettes 

f. I don’t smoke cigarettes 



The next questions are about SMOKELESS 
TOBACCO . By smokeless tobacco, we mean 
CHEWING TOBACCO OR SNUFF, including 



Redman, 


Levi Garrett, Beechnut, Skoal, Skoal 


Bandits, 


or Copenhagen. 


13. 


On how many DAYS did you use smokeless 
tobacco in the LAST MONTH f30 davsl? 




a. 


None 




b. 


1 or 2 days in the last month 




c. ’ 


3 to 5 days in the last month 




d. 


6 to 9 days in the last month 




e. 


10 to 19 days in the last month 




f. 


20 to 31 days in the last month 


14. 


On the days you used smokeless tobacco, how 
many times did you use it? 




a. 


Less than once 




b. 


1 or 2 times 




c. 


3 to 7 times 




d. 


8 to 12 times 




e. 


More than 12 times 




f. 


I don’t use smokeless tobacco 



er|c 
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19. 



15. Do you think your best friend smokes cigarettes 
or uses smokeless tobacco sometimes? 

a. Yes 

b. No 

16. If your friends found out that you smoked 
cigarettes or used smokeless tobacco, how do you 
think they’d feel? 

a. They would approve 

b. The would disapprove but still be my friends 

c. The would disapprove and stop being my 
friends 

d. They wouldn’t care 

17. How would your parents feel if they found out you 
smoked cigarettes or used smokeless tobacco 
sometimes? 

a. They would not be upset at all 

b. They would be a little upset 

c. They would be pretty upset 

d. They- would be very upset 

The next few questions are about ALCOHOL . By 
alcohol, we mean BEER. WINE. WINE COOLERS, 
GRAIN ALCOHOL. OR HARD LIQUOR . These 
questions refer to the use of alcohol for other than 
religious purposes . 

18. Do you think your best friend drinks alcohol 
sometimes? 

a. Yes 

b. No 



If your friends found out that you drank alcohol 
sometimes, how do you think they’d feel? 

a. They would approve 

b. They would disapprove but still be my 
friends 

c. They would disapprove and stop being my 
friends 

d. They wouldn’t care 

20. How would your parents feel if they found out you 
drank alcohol sometimes? 

a. They would not be upset at all 

b. They would be a little upset 

c. They would be pretty upset 

d. They would be very upset 

21. On how many DAYS did you have an alcoholic 
drink in the LAST MONTH (30 days)? (By a 
drink, we mean a can of beer, a glass of wine, a 
wine cooler, or a shot of hard liquor.) For 
example, if you drank alcohol each weekend night, 
that would be 8 days (4 weekends times 2 days 



each 


weekend). 


a. 


None 


b. 


1 or 2 days in the last month 


c. 


3 to 5 days in the last month 


d. 


6 to 9 days in the last month 


e. 


10 to 19 days in the last month 


f. 


20 to 31 days in the last month 
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26. 



22. On the days you drink alcohol, about how many 
drinks do you have? (By drink, we mean a can of 
beer, a glass of wine, a wine cooler, or a shot glass 
of hard liquor.) 

a. Less than a drink 

b. 1 drink 

c. 2 drinks 

d. 3 or more drinks 

e. I don't drink alcohol 

The next few questions are about MARITUANA . 
(Sometimes called dope, grass, weed, pot, 

Mary Jane, reefer, plant, bud or chronic.) 

23. Do you think your best friend uses marijuana 
sometimes? 

a. Yes 

b. No 

24. If your friends found out that you used marijuana 
sometimes, how do you think they’d feel? 

a. They would approve 

b. They would disapprove but still be my 
friends 

c. They would disapprove and stop being my 
friends 

d. They wouldn’t care 

25. How would your parents feel if they found out you 
used marijuana sometimes? 



On how many DAYS did you use any marijuana in 
the LAST MONTH (30 days)? For example, if 
you used marijuana each weekend night, that 
would be 8 days (4 weekends times 2 days each 
weekend). 

a. None 

b. 1 or 2 days in the last month 

c. 3 to 5 days in the last month 

d. 6 to 9 days in the last month 

e. 10 to 19 days in the last month 

f. 20 to 31 days in the last month 

27. On the days you use marijuana, how many times 
do you use it? 

a. Once a day 

b. Twice a day 

c. 3 or more times a day 

d. I don't use marijuana 

The next question is about INHALANTS, that 
is substances inhaled to get high, such as amyl 
and butyl nitrite (sometimes called poppers, snappers, 
rush, or hardware) or glue, aerosol sprays, gasoline 
or lighter fluids, ether, correction or cleaning fluids 
(inhalants are sometimes called huff, sniff, whiteout, 
and whippets) 

28. On how many DAYS did you use any inhalants in 
the LASXMQNTH (30 days)? 

a. None 

b. 1 or 2 days in the last month 

c. 3 to 5 days in the last month 

d. 6 to 9 days in the last month 

e. 10 to 19 days in the last month 

f. 20 to 31 days in the last month 



a. They would not be upset at all 

b. They would be a little upset 

c. They would be pretty upset 

d. They would be very upset 



ERIC 
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29. During the last 30 days, have you used any of the 
following on your own, that is, without a doctor 
telling you to take them? 

Circle the letter of each drug you have used 
in the last 30 days. 

a. COCAINE OR CRACK COCAINE 
(sometimes called rock, zip, dove, yea, yeo, 
or candy) 

b. HEROIN (sometimes called smack or 
black tar) 

c. HALLUCINOGENS, for example, LSD 
(sometimes called acid, hit, juice, or star) or 
peyote, mescaline, psilocybin (called 
mushrooms, shrumes, tops or fungus), 

PCP (called angel dust), or MDMA (called 
ecstasy) 

d. AMPHETAMINES (sometimes called speed, 
crank, ice, pep pills, and diet pills) 

e. TRANQUILIZERS, such as Valium, 

Quaaludes (sometimes called ludes), or 
Librium (called lib) 

f. BARBITURATES (sometimes called 
downers, yellows, reds or blues) 

g. PSYCHEDELICS (LSD, PCP, mescaline, 
peyote, psilocybin) 

h. 1 did not use any of these drugs in the last 30 
days. 

The next ten questions cover your feelings about 
and experiences with using alcohol and other drugs, 

30. Pretend your best friend offered you a drink of 
beer or wine and you did not want it. How hard 
would it be to refuse the offer? 



31. Pretend your best friend offered you a drink of 
beer or wine and you did not want it. How sure 
are you that you could say ''no”? 

a. Very sure 

b. Pretty sure 

c. A little unsure 

d. Not sure at all 

32. Pretend your best friend offered you some 
marijuana and you did not want it. How hard 
would it be to refuse the offer? 

a. Very easy 

b. Pretty easy 

c. Pretty hard 

d. Very hard 

33. Pretend your best friend offered you some 
marijuana and you did not want it. How sure are 
you that you could say "no'7 

a. Very sure 

b. Pretty sure 

c. A little unsure 

d. Not sure at all 

34. Pretend your best friend offered you some 
cocaine or some other drug and you did not want 
it How hard would it be to refuse the offer? 

a. Very easy 

b. Pretty easy 

c. Pretty hard 

d. Very hard 



a. Very easy 

b. Pretty easy 

c. Pretty hard 
^ d. Very hard 
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39. 



35. 


Pretend your best friend offered you some 
cocaine or some other drug and you did not want 




it. 


How sure are you that you could say “no”? 




a. 


Very sure 




b. 


Pretty sure 




c. 


A little unsure 




d. 


Not sure at all 


36. 


In the past year, how often did you ride in a car 
with a driver who had been drinking? 




a. 


Never 




b. 


Sometimes, but not often 




c. 


Often 




d. 


All the time 


37. 


In the past year, how often did you ride in a car 
with a driver who had been using drugs, such as 
marijuana or cocaine? 




a. 


Never 




b. 


Sometimes, but not often 




c. 


Often 




d. 


All the time 


38. 


In the past year, how often did you use alcohol 
just before or while attending school? 




a. 


Never 




b. 


Sometimes, but not often 




c. 


Often 




d. 


All the time 



In the past year, how often did you use drugs, 
such as marijuana or cocaine, just before or 
while attending school? 

a. Never 

b. Sometimes, but not often 

c. Often 

d. All the time 



The next four questions ask about your 
experiences with fighting and carrying weapons. 

40. In the past year, how many times were you in a 
physical fight? 

a. 0 times 

b. 1 time 

c. 2 or 3 times 

d. 4 or 5 times 

e. 6 or more times 

41. In the past year, on the days you had a physical 
fight, did you use alcohol or other drugs? 

a. I was never in a fight 

b. Yes, I sometimes used alcohol or other drugs 

c. Yes, I always used alcohol or other drugs 

d. No, I never used alcohol or other drugs 

42. During the past 30 days, on how many days did 
you carry a weapon such as a gun, knife, or club 
on school property? 

a. 0 days 

b. 1 day 

c. 2 or 3 days 

d. 4 or 5 days 

e. 6 or more days 



43. 


During the past 12 months, how many times has 


48. 


How old were you when you had sexual 




someone threatened or injured you with a weapon 




intercourse for the first time? 




such as a gun, knife, or club on school property? 




a. 


I have never had sexual intercourse 




a. 0 times 




b- 


Less than 12 years old 




b. 1 time 




c. 


12 years old 




c. 2 or 3 times 




d. 


13 years old 




d. 4 or 5 times 




e. 


14 years old 




e. 6 or 7 times 




f. 


15 years old 




f. 8 or 9 times 




g- 


16 years old 




g. 10 or 11 times 

h. 12 or more times 




h. 


17 or more years old 






49. 


Was the first time you had sexual intercourse an 
abusive incident? 


The next two questions ask about AIDS/HFV 
education and information. 




a. 


Yes 








b. 


No 


44. 


Have you ever been taught about AIDS/HTV 










infection in school? 


50. 


During your life, with how many people have you 
had sexual intercourse? 




a. Yes 

b. No 




a. 


I have never had sexual intercourse 




c. Not sure 




b. 


1 person 








c. 


2 people 


45. 


Have you ever talked about AIDS/HTV infection 




d. 


3 people 




with your parents or other adults in your family? 




e. 


4 people 








f. 


5 people 




a. Yes 

b. No 




g- 


6 or more people 




c. Not sure 


51. 


During the past 3 months, with how many 
people did you have sexual intercourse? 


The next twelve questions ask about 




a. 


I have never had sexual intercourse 


sexual beba vior. 




b. 


I have had sexual intercourse, but not during 










the past three months 


46. 


Have you ever had sexual intercourse? 




c. 


1 person 








d. 


2 people 




a. Yes 




e. 


3 people 




b. No 




f. 


4 people 








g- 


5 people 


47. 


Have you ever been sexually abused? 




h. 


6 or more people 




a. Yes 


52. 


Did 


you drink alcohol or use other drugs before 




b. No 




you 


had sexual intercourse the last time? 








a. 


I have never had sexual intercourse 








b. 


Yes 








c. 


No 



o 
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53. The last time you had sexual intercourse, did 
you or your partner use a condom? 





a. 


I have never had sexual intercourse 




b. 


Yes , 




c. 


No 


54. 


The last time you had sexual intercourse, what 
one method did you or your partner use to 
prevent pregnancy? (Select one response.) 




a. 


I have never had sexual intercourse 




b. 


No method was used to prevent pregnancy 




c. 


Birth control pills 




d. 


Condoms 




e. 


Withdrawal 




f. 


Some other method 




g- 


Not sure 


55. 


How many times have you been pregnant? 




a. 


0 times 




b. 


1 time 




c. 


2 or more times 




d. 


Not sure 


56. 


Are 


you pregnant now? 




a. 


Yes, I'm sure I am. 




b. 


I think I may be, but I'm not sure. 




c. 


No, I’m not. 


57. 


Have you ever been told by a doctor or nurse that 
you have a sexually transmitted disease such as 



genital herpes, genital warts, chlamydia, syphilis, 
gonorrhea, AIDS, or HTV infection? 

a. Yes 

b. No 



Sometimes people feel so depressed and hopeless 
about the future that they may consider attempting 
suicide, that is, taking some action to end their 
own life. The next four questions ask about these 
kinds of experiences in your life in the past year, 

58. During the past 12 months, did you ever 
seriously consider attempting suicide? 

a. Yes 

b. No 

59. During the past 12 months, did you make a plan 
about how you would attempt suicide? 

a. Yes 

b. No 

60. During the past 12 months, how many times did 
you actually attempt suicide? 

a. 0 times 

b. 1 time 

c. 2 or 3 times 

d. 4 or 5 times 

e. 6 or more times 

61. If you attempted suicide during the past 12 
months, did any attempt result in a injury, 
poisoning, or overdose that had to be treated by a 
doctor or nurse? 

a. I did not attempt suicide during the past 12 
months 

b. Yes 

c. No 

62. How honestly have you answered the questions 
on this survey? 

a. Very honestly 

b. Somewhat honestly 

c. Dishonestly 

This completes the questionnaire. Thank 
you for your open and honest answers to 
all of these questions. 
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CHRYSALIS CASE STUDY 
INITIAL INTERVIEW PROTOCOL 



Hello. I am (name) with Portland Public Schools Evaluation Department. I would like to talk with 
you about your participation in the Chrysalis program. Here is a brief fact sheet about the case 
study. Your responses and honesty can help us understand what works best about the program and 
how you think it can be improved for other young women. We'll meet every month and discuss some 
of your current concerns. These questions should take about 30 minutes. The information you tell 
me is confidential and your privacy will be protected. I would like to make a few notes during our 
conversation as a reminder for me; is that okay? Do you have any questions before we start? 



la. Do you have a nickname? 

b. How old are you? 

c. What is your father’s name? 

d. What is your mother’s name? 

e. Do you have any brothers or sisters? 

f. What are their names? 



What do people usually call you? 
When is your birthday? 
Occupation 
Occupation 

Ages Names 



2a. What grade are you in school? School 

b. What subjects are you taking? 

c. What subjects are easiest? 

d. What are you having trouble in? 

e. What do you like most about school? 

f. What do you like least about school? 



3a. What do you usually do after school? 

b. What do you like to do best of all? 

c. What are your hobbies? 

d. What kind of books do you like to read? 

e. What movies have you seen which you like? 

f. What clubs (teams, church groups) do you belong to? 



4. Tell me how you first learned about the Chrysalis program and started to be involved in it? 
(who, what, where, etc.) 



5. How long have you been involved in the Chrysalis program? 



6a. Suppose you could have three wishes and ask for anything you wanted, what would you ask 
for? What would you wish for most of all? (* highest priority wish) 

1 . 

2 . 

3. 

b. Suppose you could change people. How would you change yourself? 

c. How would you change your father? How would you change your mother? 
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7. If you could have your choice, how old would you like to be? Older than you are now? 
Younger? The same age? Why? 



8a. Tell me three things you definitely know are true. 

1 . 

2 . 

3. 

b. Three things you know are impossible. 

1 . 

2 . 

3. 



9. Every new program in schools offers something special to help students. So far, what is your 
impression of the Chrysalis program? What does it have to offer you? 



10. Think back as far as you can. Tell me the first thing you remember in your life. (If not 
indicated.. .How old were you when it happened?) 



1 1 . What's the scariest thing that ever happened to you? (car accident, fire, etc.) 



12a. Who do you think was the greatest person in the history of the world? Why? 
12b. Who do you think was the worst person who ever lived? Why? 



1 3. What do you do best of all? 

14. When you (leave school/are grown up/get older) what would you like to be? Why? 



1 5a. No one is perfect; everybody has some good and some bad in him, some things we like about 
people and some things we don't like. (Tell me some things you like about yourself.) 

15b. Tell me some things you do not like about yourself. (Tell me the worst thing about yourself) 



16. Do you ever feel that you would like to change places with someone? Tell me about it. 



17a. Everyone gets angry or mad sometimes. What makes you angry or lose your temper? 
17b. When you get angry, what do you do? 

17c. What do your parents or others you live with do to get you angry? 

17d. What does your brother/sister do to get you angry? 

17e. What does your teacher do to get you angry? 

17f. What do you do to make them angry at you? 
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18. Everyone feels unhappy/sad sometimes. What makes you feel that way? What sort of things 
make you cry? 

19. What would you say is your greatest worry? Tell me about it. 

20. For you, what has been the most important part of the project so tar? (your favorite event, most 
meaningful discussion, what you look forward to doing, etc.) 



FOLLOW-UP INTERVIEW PROTOCOL 

These questions will be asked of all case study program group participants, but the sequence and length of 
the interview questions and probes will be left to the discretion of the researcher. 

Theme A. How important is Chrysalis compared with other life events in influencing whether 
and how students change? 

A1 . I'd like to know about some of the things going on in your life since the last time we talked (or 
during the last 3 months) tell me what, if any, changes happened in: 

• the people you live with (household composition) 

• your best friend 

• peer group/significant relationship 

• activities (school and other) you participate in 

• school (classes, work-load, grades, schedule, attendance, attitude, effort) 

• involvement with agencies (juvenile justice, CSD) 

• health (including pregnancy) 

• work experiences (if applicable) 

A2. I see there is a lot going on in your life and Chrysalis is just one small part. Compared with 
these other things, how important do you think Chrysalis is to the way you think about things? 
(at exit-How important do you think Chrysalis is to how you've changed in the past two years?) 

A3. What are the things which have had the most influence on you (i.e., the way you think about 
things, how you've changed) during this period? 

A4. What was your personal goal(s) during your involvement with this project? To what extent do 
you feel you met those goals? 

A5. How does the program/support group/activities compare with others you may have participated 
in previously? 

A6. How would you describe the difficulties of learning to live with and recover from child 
physical/sexual abuse? For example, do you have problems with school, family, friends, 
dating, relationships, expectations for your future, etc. 

A7. What changes did you have to make in your attitude or behavior to deal with the issues 
addressed in the program (physical/sexual abuse)? 

A8. Would you recommend participation in the program to others? Why or why not? 

A9. In what ways did your involvement in this program change you? 
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Theme B. How important are the various components to the overaii Chrysaiis package? 

B1 . Can you tell me generally what kind of topics or discussions you are having in Chrysalis in the 
last few weeks? (topics, events, etc.) 

B2. What do you expect to learn from your involvement in this program? 

B3. What are the best parts of Chrysalis and’why? (prompts: What is the most fun? What did you 
learn the most from? What had the greatest impact?) 

B4. What is your current attitude towards participation in Chrysalis? What are your parents and 
friends attitudes towards your participation in Chrysalis? 

B5. If the district weren't able to offer the entire Chrysalis program, what would be the best 
components to drop? 

• support groups • open session topics • case management 

• girls empowerment • challenge course • year end celebration 

B6. How well was the program implemented in your experience? (What were the positive side 
effects? What were the negative side effects?) 

B7. What should be changed in the program to make it more useful? Would you change or adapt 
the way activities are done in your school? 



Theme C. How engaged, committed, and honest are participants with the project? 

C1. Do you think girls in your group are honest when they talk 1-to-1 with the case manager? 

C2. How open/honest are the other kids when they talk in group? Give me an example, without 
using any names, of a student who may have not been honest in group? What effect does that 
have on everyone? 

C3. Are students generally honest when they respond to Chrysalis evaluation questionnaires? If 
you think students are not honest, can you tell me why? 

C4. Can you give me an example of the kind of question we ask that a student might not want to 
respond to honestly? 

C5. Is there anything we should do differently to make students more comfortable in answering our 
questions? (prompts: administration protocols, confidentiality measures) 



Theme D. What is the best target group for Chrysalis? Why are some students more 
successful than others? 

D1 . What kinds of students do you think are likely to get the most from Chrysalis? (prompts: age, 
ethnicity, personality traits, history) 

D2. Are there some students that Chrysalis is not likely to work for? 
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D3. How well do you think Chrysalis has worked for you? What would make it better? 

D4. What is your overall impression of what you got out of the program? 

D5. To what extent are you satisfied with your success/progress in the program? 

Theme E. Serendipity 

El. Tell me something you think I don't already know about Chrysalis. 

E2. What was the most challenging part of this project for you? (most difficult or demanding) 

E3. For you, what was the most important issue you discussed at Chrysalis? 

E4. Have you ever felt tense, upset, or uncomfortable about your participation in the project? Tell 
me about it. 

E5. On a scale from 1 (low) to 5 (high), do you feel a greater sense of personal strength and 
confidence based on your participation in Chrysalis? 

E6. What unexpected things did you learn during your experience with the Chrysalis program? 

E7. What is the most successful thing about Chrysalis? 

E8. What is the least successful thing about Chrysalis? 

E9. What changes would you recommend in the program? 

El 0. Any other comments? 



Probes: 

What do you mean? 

Could you tell me more about that? 

Will you tell me what you have in mind? 

I am not sure I understand what you have in mind. 

Why do you think this is so? 

Could you tell me why you feel this way? 

What do you think causes that? 

Anything else? 

What would your mom/dad or significant parent figure think if you did that? 
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Girls Empowerment Pre-Training Questionnaire 



Name: Grade: 

School: Date: _ 



Very Very 

1 . Please rate your knowledge of these topics: Good Good Some Little 

a. physical/sexual abuse prevention O O O O 

b. date rape intervention O O O O 

c. sexual harassment O O O O 

d. domestic violence O O O O 



Can do Can do Can do Can not 



Please mark your ability to do these self-defense skills: 
a. yells 


easily 

... o 


a lot 

o 


a little 

o 


do it 

o 


b. 


hand strikes 


... o 


o 


o 


o 


c. 


kicks 


... o 


o 


o 


o 


d. 


wrist grab escapes 


... o 


o 


o 


o 


e. 


choke hold escapes 


... o 


o 


o 


o 


f. 


body grasp escapes 


... o 


o 


o 


o 


g- 


verbal assertiveness 


... o 


o 


o 


o 


Please rate your confidence if you were in these situations: 
a. walking alone to and from school 


Very 

Good 

... o 


Good 

o 


Fair 

o 


Very 

Poor 

o 


b. 


walking down the street alone 


... o 


o 


o 


o 


c. 


walking alone, having a car stop and being told to get in . . . 


.... o 


o 


o 


o 


d. 


being aware of some safe places nearby 


.... o 


o 


o 


o 


e. 


knowing how to deal with problem situations on a date . . . . 


.... o 


o 


o 


o 


f. 


knowing how to deal with problem situations on the street 


.... o 


o 


o 


o 


g- 


knowing how to deal with problem situations in the halls 
and classrooms at school 


.... o 


o 


o 


o 



THANK YOU!! 
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Project Code#: 



Girls Empowerment Feedback Form 



1 . 



3. 



4. 



As a result of the class, were you able to: 




Yes 


Some 


No 


a. gain new skills and knowledge 




..o 


o 


o 


b. gain awareness of your own strength 




..o 


o 


o 


c. feel capable, confident, and empowered 




. . o 


o 


o 


d. feel ready to practice self-defense skills on your own 




..o 


o 


o 


Please rate your knowledge of these topics: 


Very 

Good 


Good 


Some 


Very 

Little 


a. physical/sexual abuse prevention 


... o 


o 


o 


O 


b. date rape intervention 


... o 


o 


o 


O 


c. sexual harassment 


... o 


o 


o 


O 


d. domestic violence 


... o 


o 


o 


O 




Can do 


Can do 


Can do 


Can not 


Please mark your ability to do these self-defense skills: 


easily 


a lot 


a little 


do it 


a. yells 


... o 


o 


o 


o 


b. hand strikes 


... o 


o 


o 


o 


c. kicks 


... o 


o 


o 


o 


d. wrist grab escapes 


... o 


o 


o 


o 


e. choke hold escapes 


. .. o 


o 


o 


o 


f. body grasp escapes 


... o 


o 


o 


o 


g. verbal assertiveness 


... o 


o 


o 


o 


Please rate your confidence if you were in these situations: 


Very 

Good 


Good 


Fair 


Very 

Poor 


a. walking alone to and from school 


... o 


o 


o 


o 


b. walking down the street alone 


. ... o 


o 


o 


o 


c. walking alone, having a car stop and being told to get in . . . 


. ... o 


o 


o 


o 


d. being aware of some safe places nearby 


.... o 


o 


o 


o 


e. knowing how to deal with problem situations on a date . . . . 


. ... o 


o 


o 


o 


f. knowing how to deal with problem situations on the street 


.... o 


o 


o 


o 


g. knowing how to deal with problem situations in the halls 
and classrooms at school 


. ... o 


o 


o 


o 



5. What was your most favorite part of the class? 

6. What was your least favorite part of the class? : 

7. Would you like to take an advanced self-defense class? (circle one) Yes Maybe No 

Thank you for joining the Girls Empowerment class. 

Stay safe. © 
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Project Chrysalis 

Challenge Course Questionnaire (pre and post) 



Name: 




Grade: 




School: 




Date: 








cribes y 


burfeeli 


ingsl 


:6day. 




YES! 


yes 


no 


NO! 


1 . I prefer games involving some luck over games requiring pure skill 


.. O 


O 


o 


O 


2. I can learn almost anything if I set my mind to it 


.. O 


O 


o 


o 


3. Competition encourages excellence 


.. o 


O 


o 


o 


4. The extent of personal achievement is often determined by chance 


.. o 


O 


o 


o 


5. Despite my best efforts I have few worthwhile accomplishments . . 


.. o 


O 


o 


o 


6. If I need help in carrying out a plan of mine, it’s usually 

difficult to get others to help 


.. o 


O 


o 


o 


7. I find it easy to play an important part in most group situations . . . 


.. o 


O 


o 


o 


8. I always like to do my part 


.. o 


O 


o 


o 


9. I like to help other people who have helped me 


.. o 


O 


o 


o 


10. 1 know when I should take responsibility and when I shouldn’t . . . 


.. o 


O 


o 


o 
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APPENDIX F 

Evaluation Summary of Chrysalis Components: 

■ Girls Empowerment 

■ Challenge Course 



o 
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Project Chrysalis 

Evaluation of the Girls Empowerment Program, 1995-96 



This report summarizes the evaluation of the Girls Empowerment Program sponsored by the Portland 
Public Schools Chrysalis Project. The program trains young women in grades nine through twelve in self- 
defense strategies and empowerment activities designed to build skills and confidence in their ability to 
handle potentially unsafe situations. The program is targeted for adolescent women who have been 
victims of childhood physical, sexual, or psychological abuse. It developed from the Portland Police 
Department’s Womens’ Strength Program. 

The Girls Empowerment course is one of several components of Project Chrysalis. The program focuses 
on developing increased resilience in young women through training in physical safety and empowerment. 
The six-hour class curriculum is composed of: 1 ) prevention activities designed to involve and empower 
students with ownership and responsibility in a various situations, 2) linkages with community-based 
Women's Strength programs, and 3) pre and post-training evaluations designed to measure participant’s 
knowledge, skills and confidence on personal safety. 

The goal of Girls Empowerment is to develop strategies to promote personal safety in high school-aged 
young women who are survivors of childhood abuse. Specifically, the goals are: 

• Build new skills and knowledge about personal safety 

• Assist adolescent women in gaining awareness of their own strength 

• Assist adolescent women in feeling capable, confident, and empowered 

• Energize adolescent women to continue these personal safety efforts 

During the second year of the Chrysalis Project, a total of 96 female students in grades nine through 
twelve participated in the Girls Empowerment training class. This number is approximately 51% of young 
women participating in the Chrysalis intervention group. Six training classes were held between October 
1995 and February 1996. All twelve high schools offered girls empowerment training to program 
participants, their mothers and other female family members. 

The 1995-96 evaluation findings indicate that participants in Chrysalis Girls Empowerment training gained 
new skills and confidence in all 18 categories measured. Approximately 90% of the participants reported 
they learned new skills and knowledge. Over 80% of the students reported they gained new awareness 
of their own strength, felt capable and empowered. The greatest increases in knowledge were in how to 
deal with date rape situations (+.61) and prevention of physical/sexual abuse (+.53). The greatest 
increases in skills were in body grasp escapes (+.90), choke hold escapes (+.86), wrist grasp escapes 
(+.83), verbal assertiveness (+.76), and hand strikes (+.74). The greatest increase in confidence was in 
dealing with potentially dangerous encounters on the street (+.47). Sixty-six percent of the participants 
(n=56) reported they would like to take an advanced self-defense class. 



Portland Public Schools - Project Chrysalis, 1995-96 

Summary of Girls Empowerment Pre-Post Training Evaluation (N = 84) 



1. As a result of the class, were you able to: 

a. gain new skills and knowledge 
p b. gain awareness of your own strength 

c. feel capable, confident, and empowered 

d. feel ready to practice self-defense skills on your own 



Yes 


Some 


No 


Number Percent 

75 89.3 

69 82.1 

67 80.7 

59 70.2 


Number Percent 

8 19.5 

14 16.7 

16 19.3 

25 29.8 


Number Percent 
1 1.2 

1 1.2 

0 0 

0 0 



2. Wliat was your most favorite part of the class? (rank order) 



Kicks/kick variations (24) 

(knee kicks; grab kick; lower body 
kicks; groin kicks) 
Punching/striking the mats (21) 
Learning new self-defense skills (10) 
(learned how to fight; learned to do 
things; learned skills; physical 
learnings; learned new things; 
learned defense tactics; learned 
how to correctly do things; learned 
moves; learned all that's needed) 
Physical activity (8) 

Yelling (8) 



Lunch/food (6) 

Hand strikes/active hits (6) 
Everything (3) 

Discussion (3) 

Instructors, practice 
Information 
Hands on 

Information on prevention 
Beating up on someone 
Eye, throat 

Yelling to get anger out 
Everything was excellent & fun! 



3. What was your least favorite part of the class? (rank order) 



Nothing (16) 


Sharing thoughts (2) 


Kicks/kick variations (16) 


Waiting so long for lunch 


(back kick; kicking the air slowly; 


Voice hurts from yelling wrong 


kicking knees & groin; side kicks; 


Role playing 


individual kicks; some of the kicks 


Jefferson kids didn't respect us 


were kind of hard; floor kicks) 


Not enough time to smoke 


Yelling (6) 


Hand strikes 


The discussions (4) 


Morning part 


The talk from the other students (4) 


Having to wake up 


Sitting (3) 


Body grabs 


Too much exercise (tired) (2) 


Elbow 


Physical activities after eating lunch (2) 


Getting hurt 


The class ending (2) 


Punches in air 


Running (2) 


Working out 


Rude instructor (2) 


Repetitive exercises 



4. Would you like to take an advanced self-defense class? 





Number 


Percent 


Yes 


56 


65.9 


Maybe 


25 


29.4 


No 


4 


4.7 



O 
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Chrysalis Girls Empowerment, 1995-96 



I Gained New Skill 




I Feel Capable, Confident, & Empowered 



100% 







1 

strength 








< 







: 


'i.2% 


NO 

1 




Ready To Pract. My Self Defense Skills 



100 % 
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Chrysalis Girls Empowerment, 1995-96 
Mean Scores on Knowledge of Prevention 



V 4 

e 

r 

y 

G 

o 

o 

^ 3 



2 



P 
o 
o 
r 

1 

Phys/Sexual Abuse Date Rape Sexual Harassmt. Domestic Violence 
KNOWLEDGE OF PREVENTION 




March 1996 (n-96) 



Chrysalis Girls Empowerment, 1995-96 
Mean Scores for Physical Safety Skills 



Yells 

Verbal Assertiveness 
Hand Strikes 
Kicks 

Wrist Grab Escapes 
Body Grasp Escapes 
Choke Hold Escapes 




Can Not Do Can Do Easily 

March 1996 (n-96) 
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Portland Public Schools - Project Chrysalis 

Summary of Girls Empowerment Knowledge, Skills, and Confidence, 1995-96 



1^'. J ; 

,/ h /n /'V'. '/ ' '■ ' 


^ A ' ' " V. 

rtV$fyOwt ?i->^Go6d 




CV^ryGobd ' 


' -i^od ' 


' 4 ^Sdme ' ' 


cVeiyUttie' 


^ N 


Pre Mean' 


iiPostMean- 


Change 


Physical/sexual abuse 
prevention 


35.1% 45.7% 13.8% 
(33) (43) (13) 


5.3% 

(5) 


61.7% 

(58) 


34.0% 

(32) 


0.0% 

(0) 


1.1% 

(1) 


94 


3.04 


3.58 


0.53 


Date rape intervention 


25.8% 40.9% 25.8% 
(24) (38) (24) 


7.5% 

(7) 


51.6% 

(48) 


34.4% 

(32) 


11.8% 

(11) 


0.0% 

(0) 


93 


2.76 


3.37 


0.61 


Sexual harassment 


28:5% 27.5% 15.6% 
(37) (36) (15) 


6.3% 

(6) 


56.3% 

(54) 


34.4% 

(33) 


3.1% 

(3) 


1.0% 

<1) 


96 


3.04 


3.50 


0.46 


Domestic violence 


40.6% 39.6% 15.6% 
(39) (38) (15) 


2.1% : 
(2) 


68.8% 

(66) 


19.8% 

(19) 


4.2% 

(4) 


2.1% 

(2) 


96 


3.15 


3.60 


0.45 




iianCiQ V' 


'‘' o " ' • ' 

bent<ot, 

'“’■tDoH s' 


v' v': 

.-fjCartPo 


k ' 

'CfliiiOp 


'^VpaiiPpt. " 


, '..Can 'Wot. > 




"-'-PreM^n 


Post Mean '' 


'fMkk 

's'' ,■• 

vc^anae ' 


Yells . 


75.3% .10.8% 12.9% 
(70) (10) (12) 


1.1% 

(1) 


80.6% 

(75) 


14.0% 

(13) 


3.2% 

(3) 


0.0% 

(0) 


93 


3.49 


3.75 


0.26 


Hand strikes ' . 


<45a%^”S9:6^^^^^ ^ : 

(42) (18) (21) 


12.0% 

(11) 


73.9% 

(68) 


17.4% 
. (16) 


6.5% 

(6) 


0.0% 

(0) 


92 


2.86 


3.61 


0.74 


Kicks' . , y lS% 


5^.6% 22.1% .'.16.8% 
(49) (21) (16) 


7.4%;: 

(7) 


54.7% 

(52) 


28.4% 

(27) 


11.6% 

(11) 


0.0% 

(0) 


95 


3.10 


3.38 


0.28 


Wrist grab escapes 


"29.2% ' .1 1.5% ’ .332% . 
(28) (11) (32) 


21.9% 

(21) 


52.1% 

(50) 


20.8% 

(20) 


17.7% 

(17) 


3.1% 

(3) 


96 


2.40 


3.23 


0.83 


Choke hold escapes . 


2i8%-i27:2% . ' 
(19) (12) (35) 


27.7% 

(26) 


44.7% 

(42) 


23.4% 

(22) 


20.2% 

(19) 


6.4% 

(6) 


94 


2.17 


3.02 


0.86 


Bodyg^P escapes 


27-2% ; 3io.9%w 
(25) (10) (30) 


28.3% 

(26) 


51.1% 

(47) 


25.0% 

(23) 


15.2% 

(14) 


5.4% 

(5) 


92 


2.25 


3.15 


0.90 


Verbal assertiveness 


743Ml.'j:24i5%S^ 
(41) (23) (22) 


. 7.4%; 
(7) 


79.8% 

(75) 


12.8% 

(12) 


3.2% 

(3) 


0.0% 

(0) 


94 


2,96 


3.72 


0.76 




•y. *■■•(; X-: ■■■,■. v.^'; .■.<■: 

mM-x 






.^Po$l'Mean /rctiwbaM 


Walking to/from school 


48.4% 32.3% 172%, 

(45) (30) (16) 


.22% 

(2) 


67.7% 

(63) 


23.7% 

(22) 


4.3% 

(4) 


0.0% 

(0) 


93 


3.27 


■3.54 


0.27 


Walkirig alone on ^TOt 


40.9% 262% ; 18.3%' 
(38) (34) (17) 


'42% 

(4) 


:2b2® 

(56) 


:28.0% 

(26) 


7.5% 

(7) 


blo% ' 
(0) 


93 


3.14 ;■ 


3.43 


0.29 


Stranger in car stops y^^^ 


34.4% : 31 2%' : 2 1.5% ’ 
(32) (29) (20) 


:12.9% 

(12) 


43.0% 

(40) 


37.6% 

(35) 


12.9% 

(12) 


3.2% 

(3) 


93 


2.87 


3.17 


0.30 


Awareness of safe places 


M8;4% 1222% 1;8% ■ 

(45) (30) (11) 


- 7 . 5 % 

(7) 


60.2% 

(56) 


23.7% 

(22) 


10.8% 

(10) 


1.1 % 
(1) 


93 


3.22 


3.34 


0.13 


Encounters bn a date 


48.9% 26.1% -14.1% 
(45) (24) (13) 


:102% 

(10) 


63.0% 

(58) 


28.3% 

(26) 


5.4% 

(5) 


0.0% ■ 
(0) 


92 


3.13 


3.48 


0.35 


Encountersbnlhe’sirei^'S; 


32.6% a23.7% 26:i%^ 
(30) (31) (24) 


1 7.6% 
(7) 


58 J% 
(54) 


29.3% 

(27) 


7.6% 

(7) 


0.0% 

(0) 


92 


2.91 


3.38 


0.47 


Encounters at school - 


37.6% : S40.9%^'^^' ^ 
(35) (38) (14) 


6.5%! 

(6) 


67.7% 

(63) 


21;5% 

(20) 


5.4% 

(5) 


0.0% 

(0) 


93 


3.10 


3.46 


0.37 



NOTE: Means are based on a 4-point scale: Rating of knowledge is 4svery good. Ssgood, 2nsonne, and 1 avery little. Means for rating of skills are 4c>can do easily, 
Sscan do a lot, 2scan do a little, and 1 ^can not do. Means for rating of confidence are 4t3very good, 3&good, 2&fair, and 1 every poor. 
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Project Chrysalis Challenge Course, Year two 
Means and Standard Deviations on Four Protective Factor Subscaies 
Locus of Control Survey with Program Students Only 



Protective Factor 
Subscaies 


No: of 
Jt^s, 


:N 


Pretest 

Mean 


Summary Statistics: ^ . 
St.bev. N Mean St.Devj' 


ifChange 


:Personal Efficacy „ 


5 




. 2.68 ' 


0.52 


46 .. 


2.81 0.51 


0.12 


intei^sonai Cc^ = " . 




"I58V^ 


2.91 


0.24 


48: 


:= 3.16 0.20 


0.25 




58 


•■■■'3.45-' 






: V3.67, ,, 0.00 


; 0.23 


ISodalComp^ 




68 . 


.3.67 


0,07 


46 . 


. 3.76 0.07 5 
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Summary of Challenge Course Survey, Year Two 
Item Analysis of Pre and Posttest Survey Responses 



Item '-' i;-: ;■;• ■'* ■ 










•'Hno!-::* 


Summary .^atistica. 


1 prefer games involving some fuck # 


24;l% 63.0% 


, 9.3% : 


: 3.7% 1 


1 19.1% : 


, 46.8% ;i 


23:4% 


10.6% 


, 1.93 


226 


0.33 


over games requiring pure skill. 


(13) (34) 


(5) 


(2) 


(9) 


(22) 


(11) 


(5) 


(54) 


(47) 






5ae% ,36.2%? 


3.4%\ 
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NOTE: Means for items 1 and 4-6 are based on a 4-point scale (NO=4, noa3, yesa2, and YES=1 ). 
NOTE: Means for items 2, 3. and 7-10 are based on a 4-point scale (YESs4, yesa3, no=2, and NOd ). 
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